[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:05:33 dist-test-slave-rnrw krb5kdc[8450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:05:33 dist-test-slave-rnrw krb5kdc[8450](info): set up 2 sockets
Aug 07 18:05:33 dist-test-slave-rnrw krb5kdc[8450](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 18:05:34.293354 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.005s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:05:34 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:05:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:05:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:05:34.495764 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36515
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:36515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:35.087630 8464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:35.088934 8464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:35.089149 8464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:35.098824 8464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:05:35.099018 8464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:35.099120 8464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:35.099228 8464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:05:35.099315 8464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:05:35.110318 8464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:36515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:35.114025 8464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:35.115931 8464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:35.152302 8469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:35.152478 8470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:35.152379 8472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:35.153359 8464 server_base.cc:1034] running on GCE node
I20260807 18:05:35.156066 8464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:35.158229 8464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:35.159502 8464 hybrid_clock.cc:648] HybridClock initialized: now 1786125935159372 us; error 124 us; skew 500 ppm
I20260807 18:05:35.160185 8464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:35 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125935, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:35.164119 8464 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:05:35.166831 8464 webserver.cc:460] Webserver started at http://127.8.62.190:37493/ using document root <none> and password file <none>
I20260807 18:05:35.169180 8464 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:35.169405 8464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:35.169827 8464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:35.179692 8464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "216c195680cb4b478b185531b459c472"
format_stamp: "Formatted at 2026-08-07 18:05:35 on dist-test-slave-rnrw"
server_key: "134bc4fe4ac2eecd59b7f4668e1b3486"
server_key_iv: "99d576e304f12498da31e86a53c6030e"
server_key_version: "encryptionkey@0"
I20260807 18:05:35.181072 8464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "216c195680cb4b478b185531b459c472"
format_stamp: "Formatted at 2026-08-07 18:05:35 on dist-test-slave-rnrw"
server_key: "134bc4fe4ac2eecd59b7f4668e1b3486"
server_key_iv: "99d576e304f12498da31e86a53c6030e"
server_key_version: "encryptionkey@0"
I20260807 18:05:35.189565 8464 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.002s
I20260807 18:05:35.198851 8479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:35.200750 8464 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:05:35.200973 8464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "216c195680cb4b478b185531b459c472"
format_stamp: "Formatted at 2026-08-07 18:05:35 on dist-test-slave-rnrw"
server_key: "134bc4fe4ac2eecd59b7f4668e1b3486"
server_key_iv: "99d576e304f12498da31e86a53c6030e"
server_key_version: "encryptionkey@0"
I20260807 18:05:35.201221 8464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:35.220952 8464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:35.226691 8464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:35.227072 8464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:35.259415 8464 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:36515
I20260807 18:05:35.259510 8536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:36515 every 8 connection(s)
I20260807 18:05:35.263257 8464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:05:35.271049 8537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:35.273739 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 8464
I20260807 18:05:35.274161 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:05:35.277549 8442 external_mini_cluster.cc:1428] Setting key 3961eed460e8c4e7739dde4ca4311eac
I20260807 18:05:35.299422 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: Bootstrap starting.
I20260807 18:05:35.304840 8537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:35.308765 8537 log.cc:824] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:35.313203 8537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: No bootstrap required, opened a new log
Aug 07 18:05:35 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:35.329602 8537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } }
I20260807 18:05:35.330241 8537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:35.330367 8537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216c195680cb4b478b185531b459c472, State: Initialized, Role: FOLLOWER
I20260807 18:05:35.331319 8537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } }
I20260807 18:05:35.331648 8537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:05:35.331843 8537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:05:35.333400 8537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:35.337337 8537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } }
I20260807 18:05:35.338344 8537 leader_election.cc:302] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 216c195680cb4b478b185531b459c472; no voters:
I20260807 18:05:35.339238 8539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:35.287840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.8.62.190:36515)
0807 18:05:35.289163 (+ 1323us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:35.289183 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:35.293050 (+ 3867us) server_negotiation.cc:407] Connection header received
0807 18:05:35.306281 (+ 13231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:35.306309 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:35.306944 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:35.307553 (+ 609us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:35.311909 (+ 4356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:35.314671 (+ 2762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:35.316790 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:35.318109 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:35.318302 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:35.325731 (+ 7429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:35.325797 (+ 66us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:35.325854 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:35.325938 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:35.331162 (+ 5224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:35.332690 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:35.332697 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:35.332703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:35.333021 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:35.333863 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:35.333871 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:35.333877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:35.334899 (+ 1022us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:35.335349 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:35.336516 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0807 18:05:35.337014 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":481,"threads_started":1}
I20260807 18:05:35.339346 8537 leader_election.cc:288] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:05:35.339531 8542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:35.341343 8542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [term 1 LEADER]: Becoming Leader. State: Replica: 216c195680cb4b478b185531b459c472, State: Running, Role: LEADER
I20260807 18:05:35.342278 8542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } }
I20260807 18:05:35.345311 8537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:05:35.361727 8544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 216c195680cb4b478b185531b459c472. Latest consensus state: current_term: 1 leader_uuid: "216c195680cb4b478b185531b459c472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } } }
I20260807 18:05:35.361743 8543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "216c195680cb4b478b185531b459c472" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "216c195680cb4b478b185531b459c472" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36515 } } }
I20260807 18:05:35.363094 8543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:35.363091 8544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472 [sys.catalog]: This master's current role is: LEADER
W20260807 18:05:35.371006 8557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:05:35.371167 8557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:05:35.371374 8558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:05:35.374357 8558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:05:35.385313 8558 catalog_manager.cc:1347] Generated new cluster ID: 2e047dd8738147af83102dcfd89eb5fc
I20260807 18:05:35.385505 8558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:05:35.443588 8558 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:05:35.445341 8558 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:05:35.461905 8558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 216c195680cb4b478b185531b459c472: Generated new TSK 0
I20260807 18:05:35.462864 8558 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:05:35.525235 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:35.988942 8564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:35.989248 8564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:35.989467 8564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:35.997404 8564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:35.997525 8564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:35.997792 8564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:05:36.009181 8564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:36.010574 8564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:36.011852 8564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:36.026957 8572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:36.027697 8570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:36.028033 8569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:36.029210 8564 server_base.cc:1034] running on GCE node
I20260807 18:05:36.029850 8564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:36.031098 8564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:36.032455 8564 hybrid_clock.cc:648] HybridClock initialized: now 1786125936032307 us; error 60 us; skew 500 ppm
I20260807 18:05:36.032874 8564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:36 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:36.036485 8564 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:05:36.038234 8564 webserver.cc:460] Webserver started at http://127.8.62.129:46475/ using document root <none> and password file <none>
I20260807 18:05:36.038987 8564 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:36.039131 8564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:36.039489 8564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:36.042712 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "e9fc922df88044d6b2399ba7cc93a344"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "c5c2ebeddbb84a8534d94e51cbb2cc88"
server_key_iv: "3aa6b46fc0050653d83fe49bf01473fb"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.043741 8564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "e9fc922df88044d6b2399ba7cc93a344"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "c5c2ebeddbb84a8534d94e51cbb2cc88"
server_key_iv: "3aa6b46fc0050653d83fe49bf01473fb"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.050150 8564 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.002s
I20260807 18:05:36.054844 8579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:36.057061 8564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:05:36.057353 8564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "e9fc922df88044d6b2399ba7cc93a344"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "c5c2ebeddbb84a8534d94e51cbb2cc88"
server_key_iv: "3aa6b46fc0050653d83fe49bf01473fb"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.057664 8564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:36.071183 8564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:36.076306 8564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:36.076706 8564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:36.078085 8564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:36.080053 8564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:36.080243 8564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:36.080520 8564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:36.080724 8564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:05:36.138217 8564 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:42633
I20260807 18:05:36.138278 8697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:42633 every 8 connection(s)
I20260807 18:05:36.139685 8564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:05:36.149875 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 8564
I20260807 18:05:36.150275 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:05:36.151430 8442 external_mini_cluster.cc:1428] Setting key efe8c1c7f19260af1ef3647be198e6a2
Aug 07 18:05:36 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:36.172506 8700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:36.143886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:57429 (local address 127.8.62.190:36515)
0807 18:05:36.144651 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:36.144658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:36.146184 (+ 1526us) server_negotiation.cc:407] Connection header received
0807 18:05:36.148867 (+ 2683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:36.148871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:36.148950 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:36.149077 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:36.152422 (+ 3345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.153668 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.156755 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.157400 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.157562 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:36.163426 (+ 5864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:36.163437 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:36.163445 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:36.163499 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:36.168099 (+ 4600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:36.169143 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:36.169152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:36.169157 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:36.169358 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:36.169912 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:36.169921 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:36.169927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:36.170441 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:36.170631 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:36.171590 (+ 959us) server_negotiation.cc:299] Negotiation successful
0807 18:05:36.171897 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":454,"threads_started":1}
I20260807 18:05:36.173318 8701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:36.144706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36515 (local address 127.8.62.129:57429)
0807 18:05:36.145982 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:36.146055 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:36.148475 (+ 2420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:36.149725 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:36.149744 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:36.150892 (+ 1148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:36.152149 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:36.152181 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.153870 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.153874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:36.156478 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:36.156516 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.157803 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.157814 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:36.158249 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:36.159168 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:36.159206 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:36.163073 (+ 3867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:36.168324 (+ 5251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:36.168333 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:36.168350 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:36.168967 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:36.169525 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:36.169529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:36.169531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:36.169765 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:36.170622 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:36.170629 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:36.171046 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 18:05:36.171406 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 18:05:36.171881 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":554,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:05:36.175328 8698 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36515
I20260807 18:05:36.175750 8698 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:36.176776 8698 heartbeater.cc:511] Master 127.8.62.190:36515 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:05:36.181465 8499 ts_manager.cc:194] Registered new tserver with Master: e9fc922df88044d6b2399ba7cc93a344 (127.8.62.129:42633)
I20260807 18:05:36.184818 8499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:57429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:05:36.205442 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:36.669631 8704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:36.670087 8704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:36.670346 8704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:36.679992 8704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:36.680202 8704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:36.680387 8704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:05:36.692461 8704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:36.694074 8704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:36.695844 8704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:36.714713 8710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:36.714643 8711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:36.714949 8713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:36.716192 8704 server_base.cc:1034] running on GCE node
I20260807 18:05:36.717008 8704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:36.718082 8704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:36.719344 8704 hybrid_clock.cc:648] HybridClock initialized: now 1786125936719298 us; error 37 us; skew 500 ppm
I20260807 18:05:36.719794 8704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:36 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:36.728592 8704 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:05:36.730589 8704 webserver.cc:460] Webserver started at http://127.8.62.130:34749/ using document root <none> and password file <none>
I20260807 18:05:36.731489 8704 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:36.731653 8704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:36.731998 8704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:36.735457 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "5af2d643bf3e491897b07134dcc6b192"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "a7544ffe060a079545b825ce403b30d7"
server_key_iv: "38ebc55919835a5fa7e1b88b9f677044"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.736640 8704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "5af2d643bf3e491897b07134dcc6b192"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "a7544ffe060a079545b825ce403b30d7"
server_key_iv: "38ebc55919835a5fa7e1b88b9f677044"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.743073 8704 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:05:36.747210 8720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:36.748741 8704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:05:36.748890 8704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "5af2d643bf3e491897b07134dcc6b192"
format_stamp: "Formatted at 2026-08-07 18:05:36 on dist-test-slave-rnrw"
server_key: "a7544ffe060a079545b825ce403b30d7"
server_key_iv: "38ebc55919835a5fa7e1b88b9f677044"
server_key_version: "encryptionkey@0"
I20260807 18:05:36.749193 8704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:36.778173 8704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:36.784248 8704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:36.784520 8704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:36.785562 8704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:36.787277 8704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:36.787356 8704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:36.787508 8704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:36.787626 8704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:36.846398 8704 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:35129
I20260807 18:05:36.846678 8839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:35129 every 8 connection(s)
I20260807 18:05:36.848204 8704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:05:36.858095 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 8704
I20260807 18:05:36.858309 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:05:36.858801 8442 external_mini_cluster.cc:1428] Setting key 8d7e65d42c202dbf6f920fe46a111afd
Aug 07 18:05:36 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125936, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:36.873917 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:36.851513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:36623 (local address 127.8.62.190:36515)
0807 18:05:36.852206 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:36.852210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:36.853557 (+ 1347us) server_negotiation.cc:407] Connection header received
0807 18:05:36.855854 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:36.855860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:36.855961 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:36.856081 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:36.858020 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.859105 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.861315 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.861825 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.861981 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:36.866685 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:36.866691 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:36.866694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:36.866729 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:36.870329 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:36.871141 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:36.871146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:36.871152 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:36.871342 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:36.871830 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:36.871834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:36.871837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:36.872122 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:36.872285 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:36.873110 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 18:05:36.873348 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":456,"threads_started":1}
I20260807 18:05:36.875329 8843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:36.852314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36515 (local address 127.8.62.130:36623)
0807 18:05:36.853334 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:36.853427 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:36.855562 (+ 2135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:36.856358 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:36.856367 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:36.856981 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:36.857833 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:36.857853 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.859316 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.859319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:36.861164 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:36.861177 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:36.862228 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:36.862236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:36.862506 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:36.863433 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:36.863456 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:36.866420 (+ 2964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:36.870500 (+ 4080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:36.870510 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:36.870531 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:36.870998 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:36.871466 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:36.871471 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:36.871475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:36.871695 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:36.872257 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:36.872265 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:36.872719 (+ 454us) client_negotiation.cc:769] Sending connection context
0807 18:05:36.873044 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 18:05:36.873743 (+ 699us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":448,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:05:36.877182 8840 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36515
I20260807 18:05:36.877781 8840 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:36.878672 8840 heartbeater.cc:511] Master 127.8.62.190:36515 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:05:36.880494 8499 ts_manager.cc:194] Registered new tserver with Master: 5af2d643bf3e491897b07134dcc6b192 (127.8.62.130:35129)
I20260807 18:05:36.882367 8499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:36623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:05:36.906080 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:05:37.189577 8698 heartbeater.cc:503] Master 127.8.62.190:36515 was elected leader, sending a full tablet report...
W20260807 18:05:37.371335 8846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:37.371763 8846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:37.372028 8846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:37.380986 8846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:37.381122 8846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:37.381327 8846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:05:37.393699 8846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:37.395076 8846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:37.396526 8846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:37.412848 8852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:37.412917 8854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:37.413437 8851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:37.414886 8846 server_base.cc:1034] running on GCE node
I20260807 18:05:37.415645 8846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:37.417155 8846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:37.418416 8846 hybrid_clock.cc:648] HybridClock initialized: now 1786125937418345 us; error 53 us; skew 500 ppm
I20260807 18:05:37.418852 8846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:37.423561 8846 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:05:37.425537 8846 webserver.cc:460] Webserver started at http://127.8.62.131:41541/ using document root <none> and password file <none>
I20260807 18:05:37.426379 8846 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:37.426504 8846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:37.426951 8846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:37.430971 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "3fd744e4e83a4284afce58e82b3a5bf4"
format_stamp: "Formatted at 2026-08-07 18:05:37 on dist-test-slave-rnrw"
server_key: "434719931f7663d5537b3a21c6edc4b0"
server_key_iv: "fee3c8b24ab75adc916e9e9b4c51cdea"
server_key_version: "encryptionkey@0"
I20260807 18:05:37.431981 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "3fd744e4e83a4284afce58e82b3a5bf4"
format_stamp: "Formatted at 2026-08-07 18:05:37 on dist-test-slave-rnrw"
server_key: "434719931f7663d5537b3a21c6edc4b0"
server_key_iv: "fee3c8b24ab75adc916e9e9b4c51cdea"
server_key_version: "encryptionkey@0"
I20260807 18:05:37.438977 8846 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:05:37.443861 8861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:37.445571 8846 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:05:37.445818 8846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "3fd744e4e83a4284afce58e82b3a5bf4"
format_stamp: "Formatted at 2026-08-07 18:05:37 on dist-test-slave-rnrw"
server_key: "434719931f7663d5537b3a21c6edc4b0"
server_key_iv: "fee3c8b24ab75adc916e9e9b4c51cdea"
server_key_version: "encryptionkey@0"
I20260807 18:05:37.446060 8846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:37.470194 8846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:37.476022 8846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:37.476374 8846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:37.477842 8846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:37.479269 8846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:37.479346 8846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:37.479522 8846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:37.479589 8846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:37.536943 8846 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:40071
I20260807 18:05:37.537065 8981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:40071 every 8 connection(s)
I20260807 18:05:37.538798 8846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:05:37.539570 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 8846
I20260807 18:05:37.539892 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:05:37.540700 8442 external_mini_cluster.cc:1428] Setting key 696d33b9355c49ff7951100becc7ee9a
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:37.565798 8984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.542482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:55261 (local address 127.8.62.190:36515)
0807 18:05:37.543092 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.543097 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.544659 (+ 1562us) server_negotiation.cc:407] Connection header received
0807 18:05:37.547588 (+ 2929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.547592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.547662 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.547765 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.549584 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.550910 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.553159 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.553875 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.554007 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.558523 (+ 4516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.558528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.558532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.558567 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.562061 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.563126 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.563131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.563133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.563323 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.563959 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.563963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.563965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.564295 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.564412 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.565325 (+ 913us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.565508 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":310,"threads_started":1}
I20260807 18:05:37.566910 8985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.543290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36515 (local address 127.8.62.131:55261)
0807 18:05:37.544385 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:37.544470 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:37.547236 (+ 2766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:37.548035 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:37.548044 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:37.548796 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:37.549405 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:37.549423 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.551117 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.551121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:37.553020 (+ 1899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:37.553033 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.554082 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.554088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:37.554297 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.555017 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:37.555041 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:37.558122 (+ 3081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:37.562199 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:37.562206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:37.562221 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:37.562985 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:37.563455 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:37.563459 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:37.563461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:37.563835 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:37.564420 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:37.564427 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:37.564924 (+ 497us) client_negotiation.cc:769] Sending connection context
0807 18:05:37.565222 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 18:05:37.565675 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":388,"threads_started":1}
I20260807 18:05:37.568601 8982 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36515
I20260807 18:05:37.568996 8982 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:37.569788 8982 heartbeater.cc:511] Master 127.8.62.190:36515 requested a full tablet report, sending...
I20260807 18:05:37.571497 8499 ts_manager.cc:194] Registered new tserver with Master: 3fd744e4e83a4284afce58e82b3a5bf4 (127.8.62.131:40071)
I20260807 18:05:37.572708 8499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:55261
I20260807 18:05:37.584003 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:37.635968 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.617851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.8.62.129:42633)
0807 18:05:37.619132 (+ 1281us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.619144 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.619179 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:05:37.619254 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.619261 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.619508 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.619688 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.621507 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.622561 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.624100 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.624863 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.625019 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.629288 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.629301 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.629316 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.629365 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.632975 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.633698 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.633705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.633708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.633846 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.634247 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.634252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.634256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.634614 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.634950 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.635271 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.635565 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":747,"threads_started":1}
W20260807 18:05:37.637470 8608 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47804
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:37.686563 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.670758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47814 (local address 127.8.62.129:42633)
0807 18:05:37.670974 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.670980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.671140 (+ 160us) server_negotiation.cc:407] Connection header received
0807 18:05:37.671386 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.671391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.671460 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.671566 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.673086 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.673791 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.675048 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.675540 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.675680 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.680009 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.680015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.680018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.680055 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.683308 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.684115 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.684127 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.684129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.684347 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.684802 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.684806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.684808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.685171 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.685541 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.685857 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.686136 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:05:37.688653 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 8564
I20260807 18:05:37.705367 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.689777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.8.62.129:42633)
0807 18:05:37.690058 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.690063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.690304 (+ 241us) server_negotiation.cc:407] Connection header received
0807 18:05:37.690511 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.690516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.690591 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.690693 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.692259 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.693210 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.694800 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.695377 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.695521 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.698322 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.698327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.698330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.698362 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.702107 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.702923 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.702929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.702932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.703078 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.703500 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.703506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.703509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.703835 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.704179 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.704479 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.704790 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:05:37.706406 8693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:37.799038 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 8564
I20260807 18:05:37.819039 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 8704
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:05:37.835601 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.820013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.8.62.130:35129)
0807 18:05:37.820641 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.820650 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.820702 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:05:37.820791 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.820798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.821009 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.821150 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.822310 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.823307 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.824512 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.825189 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.825293 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.829773 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.829785 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.829801 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.829839 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.832700 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.833378 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.833394 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.833396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.833515 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.833936 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.833943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.833946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.834304 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.834611 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.834940 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.835230 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":334,"threads_started":1}
I20260807 18:05:37.836870 8834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:37.920122 8840 heartbeater.cc:503] Master 127.8.62.190:36515 was elected leader, sending a full tablet report...
I20260807 18:05:37.921468 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 8704
W20260807 18:05:37.946280 8481 connection.cc:535] server connection from 127.8.62.130:36623 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:05:37.946482 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 8846
Aug 07 18:05:37 dist-test-slave-rnrw krb5kdc[8450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:05:37.964852 8999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:37.947630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.8.62.131:40071)
0807 18:05:37.948204 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:37.948211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:37.948241 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:05:37.948302 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:37.948308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:37.948662 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:37.948803 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:37.950162 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.951574 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.953029 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:37.953828 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:37.953940 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:37.958507 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:37.958522 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:37.958538 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:37.958598 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:37.961721 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.962469 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.962478 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.962480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.962635 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:37.963027 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:37.963031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:37.963033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:37.963413 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:37.963700 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:37.964125 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 18:05:37.964345 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":303,"threads_started":1}
I20260807 18:05:37.966310 8977 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:38.050627 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 8846
I20260807 18:05:38.070402 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 8464
I20260807 18:05:38.071776 8532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:38.176915 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 8464
2026-08-07T18:05:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5132 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:05:38 dist-test-slave-rnrw krb5kdc[9004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:05:38 dist-test-slave-rnrw krb5kdc[9004](info): set up 2 sockets
Aug 07 18:05:38 dist-test-slave-rnrw krb5kdc[9004](info): commencing operation
krb5kdc: starting...
W20260807 18:05:39.291842 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:05:39 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:05:39.485301 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37363
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:39.825456 9018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:39.825857 9018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:39.826052 9018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:39.834321 9018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:05:39.834503 9018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:39.834610 9018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:39.834712 9018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:05:39.834805 9018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:05:39.845306 9018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:37363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:39.846769 9018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:39.848188 9018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:39.860828 9024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:39.860884 9026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:39.860780 9028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:39.861699 9018 server_base.cc:1034] running on GCE node
I20260807 18:05:39.862577 9018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:39.864241 9018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:39.865530 9018 hybrid_clock.cc:648] HybridClock initialized: now 1786125939865420 us; error 100 us; skew 500 ppm
I20260807 18:05:39.865999 9018 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:39 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125939, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:39.869464 9018 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:05:39.871240 9018 webserver.cc:460] Webserver started at http://127.8.62.190:36465/ using document root <none> and password file <none>
I20260807 18:05:39.872008 9018 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:39.872165 9018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:39.872498 9018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:39.875658 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "ee436e8e70224cfb850333b2f7758b89"
format_stamp: "Formatted at 2026-08-07 18:05:39 on dist-test-slave-rnrw"
server_key: "ca7aa237ad4c348d03085f8045867c95"
server_key_iv: "20ccb224b55d178cd29dea515b22a5bf"
server_key_version: "encryptionkey@0"
I20260807 18:05:39.876456 9018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "ee436e8e70224cfb850333b2f7758b89"
format_stamp: "Formatted at 2026-08-07 18:05:39 on dist-test-slave-rnrw"
server_key: "ca7aa237ad4c348d03085f8045867c95"
server_key_iv: "20ccb224b55d178cd29dea515b22a5bf"
server_key_version: "encryptionkey@0"
I20260807 18:05:39.882078 9018 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:05:39.885761 9035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:39.887169 9018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:05:39.887373 9018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "ee436e8e70224cfb850333b2f7758b89"
format_stamp: "Formatted at 2026-08-07 18:05:39 on dist-test-slave-rnrw"
server_key: "ca7aa237ad4c348d03085f8045867c95"
server_key_iv: "20ccb224b55d178cd29dea515b22a5bf"
server_key_version: "encryptionkey@0"
I20260807 18:05:39.887648 9018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:39.916860 9018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:39.921880 9018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:39.922223 9018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:39.942267 9018 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:37363
I20260807 18:05:39.942314 9089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:37363 every 8 connection(s)
I20260807 18:05:39.943826 9018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:05:39.946306 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9018
I20260807 18:05:39.946586 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:05:39.947443 8442 external_mini_cluster.cc:1428] Setting key e050881d87661ea7292275aa6fac56bf
I20260807 18:05:39.948815 9090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:05:39 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:39.961320 9090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89: Bootstrap starting.
I20260807 18:05:39.964911 9090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:39.966017 9090 log.cc:824] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:39.969179 9090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89: No bootstrap required, opened a new log
I20260807 18:05:39.970454 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:39.949598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45252 (local address 127.8.62.190:37363)
0807 18:05:39.950413 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:39.950426 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:39.950467 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:05:39.952096 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:39.952118 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:39.952519 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:39.953004 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:39.954495 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:39.955840 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:39.957241 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:39.957938 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:39.958163 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:39.962511 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:39.962520 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:39.962538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:39.962584 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:39.966026 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:39.966844 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:39.966852 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:39.966862 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:39.967048 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:39.967473 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:39.967483 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:39.967485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:39.968035 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:39.968362 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:39.968923 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 18:05:39.969403 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":380,"threads_started":1}
I20260807 18:05:39.977535 9090 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } }
I20260807 18:05:39.977937 9090 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:39.978060 9090 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee436e8e70224cfb850333b2f7758b89, State: Initialized, Role: FOLLOWER
I20260807 18:05:39.978657 9090 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } }
I20260807 18:05:39.978889 9090 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:05:39.979027 9090 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:05:39.979211 9090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:39.981594 9090 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } }
I20260807 18:05:39.982242 9090 leader_election.cc:302] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee436e8e70224cfb850333b2f7758b89; no voters:
I20260807 18:05:39.983008 9090 leader_election.cc:288] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:05:39.983173 9095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:39.984076 9095 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [term 1 LEADER]: Becoming Leader. State: Replica: ee436e8e70224cfb850333b2f7758b89, State: Running, Role: LEADER
I20260807 18:05:39.984740 9095 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } }
I20260807 18:05:39.985731 9090 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:05:39.987803 9097 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee436e8e70224cfb850333b2f7758b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } } }
I20260807 18:05:39.987908 9098 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee436e8e70224cfb850333b2f7758b89. Latest consensus state: current_term: 1 leader_uuid: "ee436e8e70224cfb850333b2f7758b89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee436e8e70224cfb850333b2f7758b89" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37363 } } }
I20260807 18:05:39.988330 9098 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:39.988319 9097 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:39.990691 9101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:05:39.996743 9101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:05:40.006059 9101 catalog_manager.cc:1347] Generated new cluster ID: 4556aabebc1b4ce68c13bc0ea49f3643
I20260807 18:05:40.006201 9101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:05:40.045351 9101 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:05:40.046788 9101 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:05:40.059373 9101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee436e8e70224cfb850333b2f7758b89: Generated new TSK 0
I20260807 18:05:40.060603 9101 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:05:40.118637 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:40.462664 9117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:40.462956 9117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:40.463130 9117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:40.470516 9117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:40.470626 9117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:40.470841 9117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:05:40.481137 9117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:40.482492 9117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:40.483865 9117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:40.497442 9123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:40.497574 9125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:40.498678 9122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:40.498879 9117 server_base.cc:1034] running on GCE node
I20260807 18:05:40.499536 9117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:40.500808 9117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:40.502028 9117 hybrid_clock.cc:648] HybridClock initialized: now 1786125940501981 us; error 43 us; skew 500 ppm
I20260807 18:05:40.502362 9117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:40 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:40.505999 9117 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:05:40.507583 9117 webserver.cc:460] Webserver started at http://127.8.62.129:39991/ using document root <none> and password file <none>
I20260807 18:05:40.508373 9117 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:40.508579 9117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:40.508958 9117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:40.512220 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "08a315bb84e64754a61fa036cf5699b2"
format_stamp: "Formatted at 2026-08-07 18:05:40 on dist-test-slave-rnrw"
server_key: "ab38732f0c9d4a0c6aa7bd535b691ab1"
server_key_iv: "612a155a2f0c9e975cf620b77934c8be"
server_key_version: "encryptionkey@0"
I20260807 18:05:40.513159 9117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "08a315bb84e64754a61fa036cf5699b2"
format_stamp: "Formatted at 2026-08-07 18:05:40 on dist-test-slave-rnrw"
server_key: "ab38732f0c9d4a0c6aa7bd535b691ab1"
server_key_iv: "612a155a2f0c9e975cf620b77934c8be"
server_key_version: "encryptionkey@0"
I20260807 18:05:40.518970 9117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:05:40.522702 9132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:40.524204 9117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:05:40.524497 9117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "08a315bb84e64754a61fa036cf5699b2"
format_stamp: "Formatted at 2026-08-07 18:05:40 on dist-test-slave-rnrw"
server_key: "ab38732f0c9d4a0c6aa7bd535b691ab1"
server_key_iv: "612a155a2f0c9e975cf620b77934c8be"
server_key_version: "encryptionkey@0"
I20260807 18:05:40.524789 9117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:40.548909 9117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:40.553583 9117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:40.553910 9117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:40.554975 9117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:40.556461 9117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:40.556607 9117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:40.556775 9117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:40.556842 9117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:40.594632 9117 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:36427
I20260807 18:05:40.594744 9252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:36427 every 8 connection(s)
I20260807 18:05:40.596199 9117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:05:40.600255 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9117
I20260807 18:05:40.600639 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:05:40.601419 8442 external_mini_cluster.cc:1428] Setting key 8112590526b76026408d97797143309b
Aug 07 18:05:40 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125940, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:05:40.622771 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:40.599160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:40571 (local address 127.8.62.190:37363)
0807 18:05:40.599694 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:40.599699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:40.600969 (+ 1270us) server_negotiation.cc:407] Connection header received
0807 18:05:40.603336 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:40.603342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:40.603429 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:40.603647 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:40.606189 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:40.607043 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:40.609669 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:40.610529 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:40.610710 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:40.614730 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:40.614741 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:40.614750 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:40.614803 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:40.619171 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:40.620038 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:40.620048 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:40.620052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:40.620220 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:40.620773 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:40.620781 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:40.620784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:40.621223 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:40.621391 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:40.622197 (+ 806us) server_negotiation.cc:299] Negotiation successful
0807 18:05:40.622412 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":292,"threads_started":1}
I20260807 18:05:40.623735 9256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:40.599700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37363 (local address 127.8.62.129:40571)
0807 18:05:40.600752 (+ 1052us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:40.600804 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:40.602998 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:40.603955 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:40.603969 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:40.604948 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:40.605955 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:40.605988 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:40.607247 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:40.607257 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:40.609483 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:40.609499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:40.610788 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:40.610793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:40.611034 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:40.611877 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:40.611903 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:40.614418 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:40.619362 (+ 4944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:40.619370 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:40.619385 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:40.619880 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:40.620350 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:40.620353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:40.620355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:40.620632 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:40.621374 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:40.621380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:40.621816 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 18:05:40.622055 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 18:05:40.622472 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":475,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:05:40.625872 9253 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37363
I20260807 18:05:40.626258 9253 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:40.627074 9253 heartbeater.cc:511] Master 127.8.62.190:37363 requested a full tablet report, sending...
I20260807 18:05:40.629827 9052 ts_manager.cc:194] Registered new tserver with Master: 08a315bb84e64754a61fa036cf5699b2 (127.8.62.129:36427)
I20260807 18:05:40.631412 9052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:40571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:05:40.650102 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:40.992995 9259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:40.993288 9259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:40.993417 9259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:41.001066 9259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:41.001187 9259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:41.001304 9259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:05:41.011241 9259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:41.012571 9259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:41.013880 9259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:41.027657 9267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:41.027684 9269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:41.027778 9266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:41.029333 9259 server_base.cc:1034] running on GCE node
I20260807 18:05:41.030062 9259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:41.031086 9259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:41.032348 9259 hybrid_clock.cc:648] HybridClock initialized: now 1786125941032270 us; error 68 us; skew 500 ppm
I20260807 18:05:41.032857 9259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:41 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125941, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:41.036269 9259 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:05:41.037990 9259 webserver.cc:460] Webserver started at http://127.8.62.130:40391/ using document root <none> and password file <none>
I20260807 18:05:41.038771 9259 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:41.038923 9259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:41.039253 9259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:41.042373 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "7af65db079734264b86ed50a2a1519e0"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "9c719b319701792271637a4c6449dbb0"
server_key_iv: "5b0835c83c74792ce17627c9dad27555"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.043233 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "7af65db079734264b86ed50a2a1519e0"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "9c719b319701792271637a4c6449dbb0"
server_key_iv: "5b0835c83c74792ce17627c9dad27555"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.049126 9259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 18:05:41.053118 9276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.054776 9259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:05:41.055009 9259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "7af65db079734264b86ed50a2a1519e0"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "9c719b319701792271637a4c6449dbb0"
server_key_iv: "5b0835c83c74792ce17627c9dad27555"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.055253 9259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:41.084920 9259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:41.089871 9259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:41.090209 9259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:41.091337 9259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:41.093088 9259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:41.093195 9259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.093344 9259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:41.093417 9259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.128999 9259 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:34329
I20260807 18:05:41.129041 9394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:34329 every 8 connection(s)
I20260807 18:05:41.130407 9259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:05:41.132985 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9259
I20260807 18:05:41.133232 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:05:41.133677 8442 external_mini_cluster.cc:1428] Setting key b65bb11bbd2b53085b4950664e63f19a
Aug 07 18:05:41 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125941, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:05:41.156179 9397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.133191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:56255 (local address 127.8.62.190:37363)
0807 18:05:41.134584 (+ 1393us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.134590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.135298 (+ 708us) server_negotiation.cc:407] Connection header received
0807 18:05:41.138287 (+ 2989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.138291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.138363 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.138463 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:41.141010 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.142316 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.144063 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.144986 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.145152 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.149659 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:41.149665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:41.149668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:41.149707 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:41.152612 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.153546 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.153550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.153553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.153695 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.154179 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.154191 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.154193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.154569 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:41.154735 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.155515 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.155780 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":1131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:05:41.157617 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.133919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37363 (local address 127.8.62.130:56255)
0807 18:05:41.135130 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:41.135179 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:41.137922 (+ 2743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:41.138799 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:41.138816 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:41.139753 (+ 937us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:41.140717 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.140750 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.142587 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.142591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.143872 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.143892 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.145214 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.145218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.145429 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.146273 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:41.146298 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:41.149362 (+ 3064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:41.152799 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:41.152809 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:41.152830 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:41.153366 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:41.153822 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:41.153827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:41.153831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:41.154048 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:41.154708 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:41.154717 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:41.155114 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 18:05:41.155405 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 18:05:41.156008 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":651,"threads_started":1}
I20260807 18:05:41.159547 9395 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37363
I20260807 18:05:41.159935 9395 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:41.160995 9395 heartbeater.cc:511] Master 127.8.62.190:37363 requested a full tablet report, sending...
I20260807 18:05:41.162792 9052 ts_manager.cc:194] Registered new tserver with Master: 7af65db079734264b86ed50a2a1519e0 (127.8.62.130:34329)
I20260807 18:05:41.163671 9052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:56255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:05:41.185146 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:44511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:41.553584 9401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:41.553936 9401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:41.554119 9401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:41.562343 9401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:41.562513 9401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:41.562697 9401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:05:41.572751 9401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:44511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37363
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:41.574141 9401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:41.575488 9401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:41.589519 9406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:41.589558 9409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:41.589476 9407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:41.590303 9401 server_base.cc:1034] running on GCE node
I20260807 18:05:41.591053 9401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:41.592178 9401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:41.593442 9401 hybrid_clock.cc:648] HybridClock initialized: now 1786125941593366 us; error 53 us; skew 500 ppm
I20260807 18:05:41.593852 9401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:41 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125941, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:41.597266 9401 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:05:41.598973 9401 webserver.cc:460] Webserver started at http://127.8.62.131:45679/ using document root <none> and password file <none>
I20260807 18:05:41.599716 9401 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:41.599854 9401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:41.600194 9401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:41.603430 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "d12a35f149e447f0bd35aedcf760586f"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "ab242353dc6795473d5b6e2b34b5a5f0"
server_key_iv: "6c438538beb3a867b66d79ac76df3e9b"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.604321 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "d12a35f149e447f0bd35aedcf760586f"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "ab242353dc6795473d5b6e2b34b5a5f0"
server_key_iv: "6c438538beb3a867b66d79ac76df3e9b"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.610338 9401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:05:41.614295 9417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.615721 9401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:05:41.615940 9401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "d12a35f149e447f0bd35aedcf760586f"
format_stamp: "Formatted at 2026-08-07 18:05:41 on dist-test-slave-rnrw"
server_key: "ab242353dc6795473d5b6e2b34b5a5f0"
server_key_iv: "6c438538beb3a867b66d79ac76df3e9b"
server_key_version: "encryptionkey@0"
I20260807 18:05:41.616165 9401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:41.634915 9253 heartbeater.cc:503] Master 127.8.62.190:37363 was elected leader, sending a full tablet report...
I20260807 18:05:41.635264 9401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:41.640813 9401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:41.641119 9401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:41.642179 9401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:41.643631 9401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:41.643746 9401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.643896 9401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:41.643971 9401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:41.693535 9401 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:42323
I20260807 18:05:41.693524 9534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:42323 every 8 connection(s)
I20260807 18:05:41.695331 9401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:05:41.705515 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9401
I20260807 18:05:41.705744 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:05:41.706180 8442 external_mini_cluster.cc:1428] Setting key 810e0979f64dbf6d177144011e9f8fda
Aug 07 18:05:41 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125941, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:41.721161 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.698264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:52923 (local address 127.8.62.190:37363)
0807 18:05:41.698852 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.698857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.700071 (+ 1214us) server_negotiation.cc:407] Connection header received
0807 18:05:41.702755 (+ 2684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.702760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.702836 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.702953 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:41.704761 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.705811 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.708092 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.708620 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.708800 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.713867 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:41.713880 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:41.713884 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:41.713923 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:41.717150 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.718101 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.718107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.718112 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.718271 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.718783 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.718788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.718796 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.719128 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:41.719464 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.720349 (+ 885us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.720752 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":380,"threads_started":1}
I20260807 18:05:41.722523 9538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.698966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37363 (local address 127.8.62.131:52923)
0807 18:05:41.699885 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:41.699930 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:41.702435 (+ 2505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:41.703190 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:41.703199 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:41.703899 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:41.704506 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.704525 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.706024 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.706029 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.707915 (+ 1886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.707931 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.709002 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.709013 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.709564 (+ 551us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.710469 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:41.710504 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:41.713588 (+ 3084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:41.717316 (+ 3728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:41.717327 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:41.717347 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:41.717934 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:41.718392 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:41.718397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:41.718401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:41.718642 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:41.719305 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:41.719313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:41.719889 (+ 576us) client_negotiation.cc:769] Sending connection context
0807 18:05:41.720230 (+ 341us) client_negotiation.cc:240] Negotiation successful
0807 18:05:41.720945 (+ 715us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":421,"threads_started":1}
I20260807 18:05:41.724026 9535 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37363
I20260807 18:05:41.724443 9535 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:41.725292 9535 heartbeater.cc:511] Master 127.8.62.190:37363 requested a full tablet report, sending...
I20260807 18:05:41.726750 9052 ts_manager.cc:194] Registered new tserver with Master: d12a35f149e447f0bd35aedcf760586f (127.8.62.131:42323)
I20260807 18:05:41.727789 9052 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:52923
I20260807 18:05:41.735167 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:05:41.769153 9537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.746583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.8.62.190:37363)
0807 18:05:41.746839 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.746846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.747808 (+ 962us) server_negotiation.cc:407] Connection header received
0807 18:05:41.747975 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.747979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.748125 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.748219 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:41.750186 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.750980 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.752571 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.753001 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.753102 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.762635 (+ 9533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:41.762642 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:41.762645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:41.762688 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:41.766376 (+ 3688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.767151 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.767155 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.767158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.767309 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:41.767728 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:41.767731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:41.767733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:41.768069 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:41.768266 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.768730 (+ 464us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.768914 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:05:41.779028 9052 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:41.799052 9052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:41.835022 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.822213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33196 (local address 127.8.62.130:34329)
0807 18:05:41.823443 (+ 1230us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.823456 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.824194 (+ 738us) server_negotiation.cc:407] Connection header received
0807 18:05:41.824697 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.824710 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.824985 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.825195 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:41.826147 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.827955 (+ 1808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.831461 (+ 3506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.833377 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.833529 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.833780 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.834380 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.834583 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":890,"threads_started":1}
I20260807 18:05:41.835986 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.822934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.8.62.129:36427)
0807 18:05:41.824154 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.824166 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.825115 (+ 949us) server_negotiation.cc:407] Connection header received
0807 18:05:41.825331 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.825338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.825603 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.825853 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:41.827075 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.828700 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.832473 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.834180 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.834341 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.834687 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.835477 (+ 790us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.835654 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":2858,"threads_started":1}
I20260807 18:05:41.836220 9550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.822349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:36427 (local address 127.0.0.1:52808)
0807 18:05:41.824984 (+ 2635us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:41.824998 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:41.825147 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:41.826248 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:41.826253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:41.826282 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:41.826882 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.826895 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.828863 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.828870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.832233 (+ 3363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.832253 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.834850 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.834856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.835060 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.835102 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:05:41.835202 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:05:41.835329 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2521,"mutex_wait_us":29,"thread_start_us":2358,"threads_started":1}
I20260807 18:05:41.836297 9548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.822102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:34329 (local address 127.0.0.1:33196)
0807 18:05:41.824044 (+ 1942us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:41.824090 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:41.824399 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:41.825284 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:41.825291 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:41.825318 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:41.825891 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.825908 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.828121 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.828127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.831253 (+ 3126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.831269 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.833810 (+ 2541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.833818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.834060 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.834126 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 18:05:41.834970 (+ 844us) client_negotiation.cc:240] Negotiation successful
0807 18:05:41.835221 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1659,"mutex_wait_us":370,"thread_start_us":1512,"threads_started":1}
I20260807 18:05:41.841130 9326 tablet_service.cc:1467] Processing CreateTablet for tablet 70d09a0a99b546c4a62e9bad909b3773 (DEFAULT_TABLE table=test-table [id=0f3db36ddb824939aaf19bef366d6dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:41.841789 9552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.824513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:42323 (local address 127.0.0.1:48266)
0807 18:05:41.831225 (+ 6712us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:41.831241 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:41.831395 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:41.832039 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:41.832044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:41.832070 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:41.833197 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.833213 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.836694 (+ 3481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.836698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.838441 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:41.838457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.840955 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.840962 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:41.841141 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.841183 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:05:41.841294 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 18:05:41.841445 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6606,"thread_start_us":6419,"threads_started":1}
I20260807 18:05:41.842115 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:41.825369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48266 (local address 127.8.62.131:42323)
0807 18:05:41.830240 (+ 4871us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:41.830249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:41.831352 (+ 1103us) server_negotiation.cc:407] Connection header received
0807 18:05:41.831574 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:41.831584 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:41.831830 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:41.832068 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:41.834106 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.836453 (+ 2347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.838755 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:41.840748 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:41.840861 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:41.841056 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:41.841360 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 18:05:41.841540 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4673,"thread_start_us":4382,"threads_started":1}
I20260807 18:05:41.842748 9326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d09a0a99b546c4a62e9bad909b3773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:41.847883 9183 tablet_service.cc:1467] Processing CreateTablet for tablet 70d09a0a99b546c4a62e9bad909b3773 (DEFAULT_TABLE table=test-table [id=0f3db36ddb824939aaf19bef366d6dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:41.848052 9467 tablet_service.cc:1467] Processing CreateTablet for tablet 70d09a0a99b546c4a62e9bad909b3773 (DEFAULT_TABLE table=test-table [id=0f3db36ddb824939aaf19bef366d6dfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:41.849838 9467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d09a0a99b546c4a62e9bad909b3773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:41.850106 9183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70d09a0a99b546c4a62e9bad909b3773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:41.863884 9556 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Bootstrap starting.
I20260807 18:05:41.867996 9557 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Bootstrap starting.
I20260807 18:05:41.870330 9556 tablet_bootstrap.cc:654] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:41.871578 9557 tablet_bootstrap.cc:654] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:41.871958 9556 log.cc:824] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:41.873212 9557 log.cc:824] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:41.875022 9559 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Bootstrap starting.
I20260807 18:05:41.876473 9556 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: No bootstrap required, opened a new log
I20260807 18:05:41.876906 9556 ts_tablet_manager.cc:1397] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260807 18:05:41.877532 9557 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: No bootstrap required, opened a new log
I20260807 18:05:41.877831 9557 ts_tablet_manager.cc:1397] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:05:41.879437 9559 tablet_bootstrap.cc:654] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:41.881387 9559 log.cc:824] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:41.886039 9559 tablet_bootstrap.cc:492] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: No bootstrap required, opened a new log
I20260807 18:05:41.886529 9559 ts_tablet_manager.cc:1397] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260807 18:05:41.889178 9557 raft_consensus.cc:348] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.889863 9557 raft_consensus.cc:374] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:41.890033 9557 raft_consensus.cc:729] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d12a35f149e447f0bd35aedcf760586f, State: Initialized, Role: FOLLOWER
I20260807 18:05:41.890983 9557 consensus_queue.cc:260] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.892935 9535 heartbeater.cc:503] Master 127.8.62.190:37363 was elected leader, sending a full tablet report...
I20260807 18:05:41.893918 9556 raft_consensus.cc:348] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.895246 9556 raft_consensus.cc:374] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:41.895422 9556 raft_consensus.cc:729] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af65db079734264b86ed50a2a1519e0, State: Initialized, Role: FOLLOWER
I20260807 18:05:41.895819 9557 ts_tablet_manager.cc:1428] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Time spent starting tablet: real 0.018s user 0.007s sys 0.008s
I20260807 18:05:41.896415 9556 consensus_queue.cc:260] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.900452 9556 ts_tablet_manager.cc:1428] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Time spent starting tablet: real 0.023s user 0.011s sys 0.007s
I20260807 18:05:41.900482 9395 heartbeater.cc:503] Master 127.8.62.190:37363 was elected leader, sending a full tablet report...
I20260807 18:05:41.901043 9559 raft_consensus.cc:348] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.901644 9559 raft_consensus.cc:374] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:41.901813 9559 raft_consensus.cc:729] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a315bb84e64754a61fa036cf5699b2, State: Initialized, Role: FOLLOWER
I20260807 18:05:41.902621 9559 consensus_queue.cc:260] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:41.904709 9559 ts_tablet_manager.cc:1428] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Time spent starting tablet: real 0.018s user 0.009s sys 0.009s
W20260807 18:05:41.947414 9536 tablet.cc:2369] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:42.062084 9564 raft_consensus.cc:482] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:42.062412 9564 raft_consensus.cc:504] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:42.064603 9564 leader_election.cc:288] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 08a315bb84e64754a61fa036cf5699b2 (127.8.62.129:36427), d12a35f149e447f0bd35aedcf760586f (127.8.62.131:42323)
I20260807 18:05:42.075060 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.065116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:44405 (local address 127.8.62.129:36427)
0807 18:05:42.065421 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.065427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.067363 (+ 1936us) server_negotiation.cc:407] Connection header received
0807 18:05:42.067721 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.067726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.067802 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.067898 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:42.069176 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.070197 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.072992 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.073944 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.074197 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.074294 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.074630 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.074765 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:05:42.075199 9566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.065607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:36427 (local address 127.8.62.130:44405)
0807 18:05:42.067222 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:42.067262 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:42.067508 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:42.067918 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:42.067923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:42.067947 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:42.068994 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:42.069005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.070355 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.070360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:42.072789 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:42.072807 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.074277 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.074284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:42.074445 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.074472 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:05:42.074581 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:05:42.074739 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1471,"mutex_wait_us":76,"spinlock_wait_cycles":67840,"thread_start_us":907,"threads_started":1}
I20260807 18:05:42.075832 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.065224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:33515 (local address 127.8.62.131:42323)
0807 18:05:42.065516 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.065522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.067304 (+ 1782us) server_negotiation.cc:407] Connection header received
0807 18:05:42.067624 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.067635 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.067704 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.067811 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:42.069110 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.070285 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.073011 (+ 2726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.074446 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.074703 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.074828 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.075447 (+ 619us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.075563 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:05:42.075984 9568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.064988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:42323 (local address 127.8.62.130:33515)
0807 18:05:42.067168 (+ 2180us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:42.067187 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:42.067434 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:42.067828 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:42.067833 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:42.067861 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:42.068913 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:42.068927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.070458 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.070465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:42.072819 (+ 2354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:42.072834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.074835 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.074844 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:42.075021 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.075048 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:05:42.075387 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 18:05:42.075543 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1983,"mutex_wait_us":55,"thread_start_us":1482,"threads_started":1}
I20260807 18:05:42.076740 9205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d09a0a99b546c4a62e9bad909b3773" candidate_uuid: "7af65db079734264b86ed50a2a1519e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a315bb84e64754a61fa036cf5699b2" is_pre_election: true
I20260807 18:05:42.077764 9205 raft_consensus.cc:2454] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af65db079734264b86ed50a2a1519e0 in term 0.
I20260807 18:05:42.077482 9489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d09a0a99b546c4a62e9bad909b3773" candidate_uuid: "7af65db079734264b86ed50a2a1519e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12a35f149e447f0bd35aedcf760586f" is_pre_election: true
I20260807 18:05:42.078382 9489 raft_consensus.cc:2454] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af65db079734264b86ed50a2a1519e0 in term 0.
I20260807 18:05:42.078903 9279 leader_election.cc:302] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a315bb84e64754a61fa036cf5699b2, 7af65db079734264b86ed50a2a1519e0; no voters:
I20260807 18:05:42.079574 9564 raft_consensus.cc:2781] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:42.079743 9564 raft_consensus.cc:482] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:42.079871 9564 raft_consensus.cc:3037] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:42.082314 9564 raft_consensus.cc:504] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
I20260807 18:05:42.083189 9564 leader_election.cc:288] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [CANDIDATE]: Term 1 election: Requested vote from peers 08a315bb84e64754a61fa036cf5699b2 (127.8.62.129:36427), d12a35f149e447f0bd35aedcf760586f (127.8.62.131:42323)
I20260807 18:05:42.083992 9205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d09a0a99b546c4a62e9bad909b3773" candidate_uuid: "7af65db079734264b86ed50a2a1519e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08a315bb84e64754a61fa036cf5699b2"
I20260807 18:05:42.084059 9489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70d09a0a99b546c4a62e9bad909b3773" candidate_uuid: "7af65db079734264b86ed50a2a1519e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d12a35f149e447f0bd35aedcf760586f"
I20260807 18:05:42.084347 9489 raft_consensus.cc:3037] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:42.084342 9205 raft_consensus.cc:3037] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:42.088425 9205 raft_consensus.cc:2454] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af65db079734264b86ed50a2a1519e0 in term 1.
I20260807 18:05:42.088594 9489 raft_consensus.cc:2454] T 70d09a0a99b546c4a62e9bad909b3773 P d12a35f149e447f0bd35aedcf760586f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af65db079734264b86ed50a2a1519e0 in term 1.
I20260807 18:05:42.089188 9279 leader_election.cc:302] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08a315bb84e64754a61fa036cf5699b2, 7af65db079734264b86ed50a2a1519e0; no voters:
I20260807 18:05:42.089848 9564 raft_consensus.cc:2781] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:42.090966 9564 raft_consensus.cc:686] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [term 1 LEADER]: Becoming Leader. State: Replica: 7af65db079734264b86ed50a2a1519e0, State: Running, Role: LEADER
I20260807 18:05:42.091678 9564 consensus_queue.cc:237] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } }
W20260807 18:05:42.099150 9254 tablet.cc:2369] T 70d09a0a99b546c4a62e9bad909b3773 P 08a315bb84e64754a61fa036cf5699b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:42.101886 9050 catalog_manager.cc:5495] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af65db079734264b86ed50a2a1519e0 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "7af65db079734264b86ed50a2a1519e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af65db079734264b86ed50a2a1519e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 34329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "08a315bb84e64754a61fa036cf5699b2" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d12a35f149e447f0bd35aedcf760586f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42323 } health_report { overall_health: UNKNOWN } } }
W20260807 18:05:42.132977 9396 tablet.cc:2369] T 70d09a0a99b546c4a62e9bad909b3773 P 7af65db079734264b86ed50a2a1519e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:42.144912 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.125451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.8.62.129:36427)
0807 18:05:42.125678 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.125683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.126239 (+ 556us) server_negotiation.cc:407] Connection header received
0807 18:05:42.126375 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.126379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.126521 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.126640 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.129261 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.130263 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.132309 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.132842 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.132994 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.137785 (+ 4791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.137801 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.137819 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.137857 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.141569 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.142331 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.142339 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.142341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.142551 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.142967 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.142975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.142978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.143441 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.143907 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.144142 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.144408 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:42.203933 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.185687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.8.62.129:36427)
0807 18:05:42.185915 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.185921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.186485 (+ 564us) server_negotiation.cc:407] Connection header received
0807 18:05:42.186764 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.186768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.186838 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.186948 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.189001 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.189863 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.191558 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.192088 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.192267 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.197097 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.197102 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.197105 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.197156 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.200863 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.201600 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.201606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.201611 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.201894 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.202352 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.202357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.202362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.202740 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.203022 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.203335 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.203582 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:42.258546 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.241291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.8.62.129:36427)
0807 18:05:42.241499 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.241504 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.241525 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:05:42.241624 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.241627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.241690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.241779 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.243626 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.244332 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.245720 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.246142 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.246248 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.251804 (+ 5556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.251810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.251813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.251854 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.255158 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.256075 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.256080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.256082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.256303 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.256822 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.256826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.256828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.257320 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.257551 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.257965 (+ 414us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.258210 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:05:42.261073 9162 tablet_service.cc:2908] Rejecting scan request for tablet 70d09a0a99b546c4a62e9bad909b3773: Uninitialized: clean time has not yet been initialized
I20260807 18:05:42.265784 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9117
I20260807 18:05:42.281679 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.266799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.8.62.129:36427)
0807 18:05:42.267037 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.267045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.267301 (+ 256us) server_negotiation.cc:407] Connection header received
0807 18:05:42.267415 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.267419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.267533 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.267636 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.269222 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.270227 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.271495 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.272143 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.272274 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.275597 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.275603 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.275606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.275658 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.278748 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.279456 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.279462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.279466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.279626 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.280020 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.280024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.280027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.280378 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.280789 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.281084 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.281283 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:05:42.282411 9248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:42.381151 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9117
I20260807 18:05:42.400931 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9259
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:05:42.417294 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.401785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.8.62.130:34329)
0807 18:05:42.402285 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.402289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.402307 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:05:42.402362 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.402366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.402481 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.402591 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.403879 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.404709 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.406188 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.406587 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.406702 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.410939 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.410954 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.410970 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.411039 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.414380 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.415154 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.415162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.415164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.415315 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.415654 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.415658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.415660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.416063 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.416304 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.416758 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.416978 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":284,"threads_started":1}
I20260807 18:05:42.418068 9390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:42.521112 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9259
I20260807 18:05:42.539343 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9401
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:05:42.555342 9554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:42.540155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.8.62.131:42323)
0807 18:05:42.540372 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:42.540381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:42.540404 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:05:42.540686 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:42.540699 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:42.540778 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:42.540891 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:42.542171 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.542877 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.544127 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:42.544520 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:42.544855 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:42.549556 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:42.549579 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:42.549606 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:42.549645 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:42.552394 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.553182 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.553191 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.553193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.553327 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:42.553733 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:42.553737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:42.553739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:42.554104 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:42.554409 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:42.554734 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 18:05:42.554994 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:05:42.556034 9530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:42.643772 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9401
I20260807 18:05:42.661237 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9018
I20260807 18:05:42.662595 9085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:42.776299 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9018
2026-08-07T18:05:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4600 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9589](info): set up 2 sockets
Aug 07 18:05:42 dist-test-slave-rnrw krb5kdc[9589](info): commencing operation
krb5kdc: starting...
W20260807 18:05:43.887925 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:05:44 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:05:44.088383 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:41681
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:41681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:44.438153 9604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:44.438480 9604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:44.438621 9604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:44.446262 9604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:05:44.446460 9604 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:05:44.446537 9604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:44.446870 9604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:44.446986 9604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:05:44.447206 9604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:05:44.458189 9604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:41681
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:41681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:44.459671 9604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:44.461124 9604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:44.473949 9612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:44.474028 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:44.473881 9609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:44.475118 9604 server_base.cc:1034] running on GCE node
I20260807 18:05:44.476006 9604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:44.477607 9604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:44.478847 9604 hybrid_clock.cc:648] HybridClock initialized: now 1786125944478783 us; error 34 us; skew 500 ppm
I20260807 18:05:44.479188 9604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:44 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:44.482547 9604 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:05:44.484081 9604 webserver.cc:460] Webserver started at http://127.8.62.190:45303/ using document root <none> and password file <none>
I20260807 18:05:44.484916 9604 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:44.485047 9604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:44.485404 9604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:44.488600 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034"
format_stamp: "Formatted at 2026-08-07 18:05:44 on dist-test-slave-rnrw"
server_key: "5d1b4741d61f2b4cac5179d2262c2c63"
server_key_iv: "81f8a314373094482dfb1eed97dee51c"
server_key_version: "encryptionkey@0"
I20260807 18:05:44.489477 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034"
format_stamp: "Formatted at 2026-08-07 18:05:44 on dist-test-slave-rnrw"
server_key: "5d1b4741d61f2b4cac5179d2262c2c63"
server_key_iv: "81f8a314373094482dfb1eed97dee51c"
server_key_version: "encryptionkey@0"
I20260807 18:05:44.494971 9604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:05:44.498698 9619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:44.500207 9604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:05:44.500427 9604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034"
format_stamp: "Formatted at 2026-08-07 18:05:44 on dist-test-slave-rnrw"
server_key: "5d1b4741d61f2b4cac5179d2262c2c63"
server_key_iv: "81f8a314373094482dfb1eed97dee51c"
server_key_version: "encryptionkey@0"
I20260807 18:05:44.500769 9604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:44.518285 9604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:44.522866 9604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:44.523159 9604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:44.546110 9604 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:41681
I20260807 18:05:44.546144 9682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:41681 every 8 connection(s)
I20260807 18:05:44.547602 9604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:05:44.548667 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9604
I20260807 18:05:44.548934 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:05:44.549611 8442 external_mini_cluster.cc:1428] Setting key 77316d6bfc350166867b53f80c060649
I20260807 18:05:44.552280 9683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:05:44 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:44.566437 9683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Bootstrap starting.
I20260807 18:05:44.570964 9683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:44.572326 9683 log.cc:824] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:44.576099 9683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: No bootstrap required, opened a new log
I20260807 18:05:44.576215 9687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:44.551724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.8.62.190:41681)
0807 18:05:44.552573 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:44.552589 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:44.552651 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:05:44.555116 (+ 2465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:44.555146 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:44.555682 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:44.556188 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:44.557827 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:44.559917 (+ 2090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:44.561205 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:44.562253 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:44.562464 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:44.567000 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:44.567011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:44.567032 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:44.567089 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:44.571238 (+ 4149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:44.572001 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:44.572012 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:44.572023 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:44.572243 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:44.572841 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:44.572847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:44.572850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:44.573545 (+ 695us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:44.573779 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:44.574351 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 18:05:44.574809 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":333,"threads_started":1}
I20260807 18:05:44.584427 9683 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } }
I20260807 18:05:44.584896 9683 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:44.585019 9683 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa1740bf1f0b480b9d1bc3e1b3e32034, State: Initialized, Role: FOLLOWER
I20260807 18:05:44.585633 9683 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } }
I20260807 18:05:44.585942 9683 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:05:44.586133 9683 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:05:44.586335 9683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:44.588732 9683 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } }
I20260807 18:05:44.589285 9683 leader_election.cc:302] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa1740bf1f0b480b9d1bc3e1b3e32034; no voters:
I20260807 18:05:44.589938 9683 leader_election.cc:288] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:05:44.590116 9689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:44.591715 9689 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [term 1 LEADER]: Becoming Leader. State: Replica: fa1740bf1f0b480b9d1bc3e1b3e32034, State: Running, Role: LEADER
I20260807 18:05:44.592383 9683 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:05:44.592336 9689 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } }
I20260807 18:05:44.595603 9691 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa1740bf1f0b480b9d1bc3e1b3e32034. Latest consensus state: current_term: 1 leader_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } } }
I20260807 18:05:44.596040 9691 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:44.596663 9690 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1740bf1f0b480b9d1bc3e1b3e32034" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41681 } } }
I20260807 18:05:44.597000 9690 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:44.597239 9700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:05:44.602438 9700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:05:44.612607 9700 catalog_manager.cc:1347] Generated new cluster ID: 7ca4100c624d47cba07a8e3cc2bd299c
I20260807 18:05:44.612839 9700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:05:44.627113 9700 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:05:44.628620 9700 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:05:44.643553 9700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Generated new TSK 0
I20260807 18:05:44.644604 9700 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:05:44.706045 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41681
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:45.059185 9713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:45.059507 9713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:45.059689 9713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:45.067135 9713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:45.067256 9713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:45.067447 9713 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:45.067581 9713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:05:45.078480 9713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:41681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:45.079960 9713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:45.081489 9713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:45.095942 9721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:45.095965 9718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:45.095912 9719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:45.096928 9713 server_base.cc:1034] running on GCE node
I20260807 18:05:45.097776 9713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:45.098920 9713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:45.100247 9713 hybrid_clock.cc:648] HybridClock initialized: now 1786125945100171 us; error 43 us; skew 500 ppm
I20260807 18:05:45.100675 9713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:45 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125945, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:45.104046 9713 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:05:45.105820 9713 webserver.cc:460] Webserver started at http://127.8.62.129:37217/ using document root <none> and password file <none>
I20260807 18:05:45.106720 9713 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:45.106846 9713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:45.107218 9713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:45.110453 9713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "443e22b7d43341bbbc24961a118b0c0c"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "1035ca84d0247c18cf878c014f9d13cd"
server_key_iv: "e672392de4d2f25562c5648574ce9f64"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.111328 9713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "443e22b7d43341bbbc24961a118b0c0c"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "1035ca84d0247c18cf878c014f9d13cd"
server_key_iv: "e672392de4d2f25562c5648574ce9f64"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.117054 9713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 18:05:45.120864 9728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:45.122498 9713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:05:45.122700 9713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "443e22b7d43341bbbc24961a118b0c0c"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "1035ca84d0247c18cf878c014f9d13cd"
server_key_iv: "e672392de4d2f25562c5648574ce9f64"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.122969 9713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:45.149564 9713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:45.154305 9713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:45.154639 9713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:45.157907 9713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:45.158038 9713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:45.158215 9713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:45.158304 9713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:45 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125945, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:45.191681 9738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.159506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.129:48749)
0807 18:05:45.160698 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:45.160760 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:45.164515 (+ 3755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:45.165846 (+ 1331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:45.165863 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:45.167078 (+ 1215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:45.168390 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.168428 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.169747 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.169757 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.172985 (+ 3228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.173008 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.174431 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.174446 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.174811 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.175825 (+ 1014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:45.175873 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:45.180281 (+ 4408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:45.185012 (+ 4731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.185031 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.185060 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.186043 (+ 983us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.186544 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.186552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.186567 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.186941 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.187568 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:45.187580 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:45.188269 (+ 689us) client_negotiation.cc:769] Sending connection context
0807 18:05:45.188717 (+ 448us) client_negotiation.cc:240] Negotiation successful
0807 18:05:45.189513 (+ 796us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":480,"threads_started":1}
I20260807 18:05:45.193396 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.158538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:48749 (local address 127.8.62.190:41681)
0807 18:05:45.161783 (+ 3245us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:45.161788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:45.161809 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:05:45.164982 (+ 3173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:45.164997 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:45.165103 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:45.165187 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:45.168743 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.169504 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.173263 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.174017 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.174225 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.180832 (+ 6607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:45.180843 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:45.180849 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:45.180905 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:45.184821 (+ 3916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.186242 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.186254 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.186257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.186402 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.187100 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.187104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.187106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.187401 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:45.187558 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:45.192513 (+ 4955us) server_negotiation.cc:299] Negotiation successful
0807 18:05:45.192903 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3128,"thread_start_us":3021,"threads_started":1}
I20260807 18:05:45.211241 9713 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:39755
I20260807 18:05:45.211349 9853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:39755 every 8 connection(s)
I20260807 18:05:45.212699 9713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:05:45.215253 9854 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41681
I20260807 18:05:45.215544 9854 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:45.216408 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.200007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:41783 (local address 127.8.62.190:41681)
0807 18:05:45.200237 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:45.200246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:45.200992 (+ 746us) server_negotiation.cc:407] Connection header received
0807 18:05:45.201265 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:45.201272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:45.201358 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:45.201448 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:45.203046 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.203810 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.206811 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.207445 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.207597 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.210029 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:45.210035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:45.210038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:45.210103 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:45.213269 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.214224 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.214230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.214234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.214402 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.214920 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.214926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.214929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.215342 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:45.215517 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:45.215969 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 18:05:45.216163 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:05:45.216451 9854 heartbeater.cc:511] Master 127.8.62.190:41681 requested a full tablet report, sending...
I20260807 18:05:45.216837 9835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.199976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.129:41783)
0807 18:05:45.200863 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:45.200894 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:45.201102 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:45.201535 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:45.201543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:45.202360 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:45.202860 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.202876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.204001 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.204009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.206633 (+ 2624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.206655 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.207707 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.207718 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.207968 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.208945 (+ 977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:45.208969 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:45.209767 (+ 798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:45.213425 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.213431 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.213435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.214097 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.214576 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.214581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.214584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.214752 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.215478 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:45.215483 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:45.215718 (+ 235us) client_negotiation.cc:769] Sending connection context
0807 18:05:45.215882 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:05:45.216269 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":426,"threads_started":1}
I20260807 18:05:45.218741 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9713
I20260807 18:05:45.218973 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:05:45.219059 9636 ts_manager.cc:194] Registered new tserver with Master: 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755)
I20260807 18:05:45.219736 8442 external_mini_cluster.cc:1428] Setting key 3a1fe0aefa0e5632e5ada62b65b739e7
I20260807 18:05:45.221141 9636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:48749
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:05:45.264923 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41681
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:45.606129 9858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:45.606423 9858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:45.606590 9858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:45.614084 9858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:45.614197 9858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:45.614358 9858 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:45.614451 9858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:05:45.625116 9858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:41681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:45.626760 9858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:45.628260 9858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:45.642885 9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:45.642933 9864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:45.642849 9863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:45.644563 9858 server_base.cc:1034] running on GCE node
I20260807 18:05:45.645257 9858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:45.646304 9858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:45.647562 9858 hybrid_clock.cc:648] HybridClock initialized: now 1786125945647490 us; error 53 us; skew 500 ppm
I20260807 18:05:45.647905 9858 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:45 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125945, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:45.651276 9858 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:05:45.653028 9858 webserver.cc:460] Webserver started at http://127.8.62.130:35353/ using document root <none> and password file <none>
I20260807 18:05:45.653772 9858 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:45.653908 9858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:45.654249 9858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:45.657320 9858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "ee1c989575ee4f51a81e9a00115c6820"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "f7e74bddb7da903b0c74bd5d74364a11"
server_key_iv: "f4f3765b133f5a457c21281b656f67a7"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.658239 9858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "ee1c989575ee4f51a81e9a00115c6820"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "f7e74bddb7da903b0c74bd5d74364a11"
server_key_iv: "f4f3765b133f5a457c21281b656f67a7"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.663801 9858 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:05:45.667716 9873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:45.669199 9858 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:05:45.669384 9858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "ee1c989575ee4f51a81e9a00115c6820"
format_stamp: "Formatted at 2026-08-07 18:05:45 on dist-test-slave-rnrw"
server_key: "f7e74bddb7da903b0c74bd5d74364a11"
server_key_iv: "f4f3765b133f5a457c21281b656f67a7"
server_key_version: "encryptionkey@0"
I20260807 18:05:45.669625 9858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:45.690294 9858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:45.695111 9858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:45.695405 9858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:45.698256 9858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:45.698374 9858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:45.698577 9858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:45.698674 9858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:45 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125945, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:45.724800 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.699185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:34621 (local address 127.8.62.190:41681)
0807 18:05:45.699389 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:45.699397 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:45.700918 (+ 1521us) server_negotiation.cc:407] Connection header received
0807 18:05:45.703067 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:45.703080 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:45.703182 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:45.703295 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:45.706504 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.707481 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.710914 (+ 3433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.711506 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.711668 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.717273 (+ 5605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:45.717279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:45.717282 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:45.717319 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:45.720671 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.721749 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.721755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.721759 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.721887 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.722529 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.722537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.722540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.722943 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:45.723110 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:45.724171 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0807 18:05:45.724406 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:05:45.726313 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.699799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.130:34621)
0807 18:05:45.700724 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:45.700775 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:45.702751 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:45.703806 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:45.703824 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:45.705011 (+ 1187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:45.706255 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.706292 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.707661 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.707670 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.710652 (+ 2982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.710672 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.712102 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.712115 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.712474 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.713422 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:45.713456 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:45.717003 (+ 3547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:45.720826 (+ 3823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.720836 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.720855 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.721607 (+ 752us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.722010 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.722015 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.722022 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.722356 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.723188 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:45.723194 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:45.723760 (+ 566us) client_negotiation.cc:769] Sending connection context
0807 18:05:45.724066 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 18:05:45.724696 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":381,"threads_started":1}
I20260807 18:05:45.741917 9858 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:36317
I20260807 18:05:45.741955 9998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:36317 every 8 connection(s)
I20260807 18:05:45.743618 9858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:05:45.744906 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 9858
I20260807 18:05:45.745141 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:05:45.745738 8442 external_mini_cluster.cc:1428] Setting key ddcd61f79df0ba11265e97775e1c603b
I20260807 18:05:45.746359 10000 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41681
I20260807 18:05:45.746667 10000 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:45.748049 10000 heartbeater.cc:511] Master 127.8.62.190:41681 requested a full tablet report, sending...
I20260807 18:05:45.751084 9636 ts_manager.cc:194] Registered new tserver with Master: ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317)
I20260807 18:05:45.751981 9636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:34621
I20260807 18:05:45.754407 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.734705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:49589 (local address 127.8.62.190:41681)
0807 18:05:45.734902 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:45.734911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:45.735935 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 18:05:45.736308 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:45.736315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:45.736421 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:45.736510 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:45.738597 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.739923 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.742439 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.742890 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.743031 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.746178 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:45.746189 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:45.746225 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:45.746306 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:45.751243 (+ 4937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.752054 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.752060 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.752064 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.752279 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:45.752794 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:45.752799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:45.752802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:45.753195 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:45.753367 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:45.753826 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:05:45.754068 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:05:45.754607 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:45.733986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.130:49589)
0807 18:05:45.735790 (+ 1804us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:45.735820 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:45.736103 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:45.736823 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:45.736831 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:45.737733 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:45.738354 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.738377 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.740136 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.740144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.742257 (+ 2113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:45.742272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:45.743132 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:45.743144 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:45.743324 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:45.744209 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:45.744247 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:45.745838 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:45.751446 (+ 5608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.751452 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.751456 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.751898 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.752407 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:45.752413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:45.752417 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:45.752639 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:45.753326 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:45.753332 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:45.753561 (+ 229us) client_negotiation.cc:769] Sending connection context
0807 18:05:45.753715 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 18:05:45.754111 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1564,"thread_start_us":926,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:05:45.793118 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41681
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:46.135286 10004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:46.135671 10004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:46.135862 10004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:46.143389 10004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:46.143575 10004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:46.143748 10004 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:46.143895 10004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:05:46.154346 10004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:41681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:46.155788 10004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:46.157289 10004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:46.172653 10009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:46.172731 10010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:46.172663 10012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:46.174088 10004 server_base.cc:1034] running on GCE node
I20260807 18:05:46.174891 10004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:46.176103 10004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:46.177413 10004 hybrid_clock.cc:648] HybridClock initialized: now 1786125946177282 us; error 116 us; skew 500 ppm
I20260807 18:05:46.177832 10004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:46 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:46.181463 10004 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:05:46.183176 10004 webserver.cc:460] Webserver started at http://127.8.62.131:43933/ using document root <none> and password file <none>
I20260807 18:05:46.183964 10004 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:46.184114 10004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:46.184479 10004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:46.187677 10004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "1dffa52f291040939ceac4498abbedb5"
format_stamp: "Formatted at 2026-08-07 18:05:46 on dist-test-slave-rnrw"
server_key: "70d60e6b31cc19839c0902001daa14e8"
server_key_iv: "93f8a90fdf7d10f34023d77aebbd94b7"
server_key_version: "encryptionkey@0"
I20260807 18:05:46.188614 10004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "1dffa52f291040939ceac4498abbedb5"
format_stamp: "Formatted at 2026-08-07 18:05:46 on dist-test-slave-rnrw"
server_key: "70d60e6b31cc19839c0902001daa14e8"
server_key_iv: "93f8a90fdf7d10f34023d77aebbd94b7"
server_key_version: "encryptionkey@0"
I20260807 18:05:46.194654 10004 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:05:46.198869 10019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:46.200631 10004 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:05:46.200832 10004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "1dffa52f291040939ceac4498abbedb5"
format_stamp: "Formatted at 2026-08-07 18:05:46 on dist-test-slave-rnrw"
server_key: "70d60e6b31cc19839c0902001daa14e8"
server_key_iv: "93f8a90fdf7d10f34023d77aebbd94b7"
server_key_version: "encryptionkey@0"
I20260807 18:05:46.201128 10004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:46.223266 10004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:46.224936 9854 heartbeater.cc:503] Master 127.8.62.190:41681 was elected leader, sending a full tablet report...
I20260807 18:05:46.228730 10004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:46.229035 10004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:46.232069 10004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:46.232164 10004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:46.232357 10004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:46.232414 10004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:46 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:46.261022 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.232769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:47381 (local address 127.8.62.190:41681)
0807 18:05:46.232988 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.232994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.234546 (+ 1552us) server_negotiation.cc:407] Connection header received
0807 18:05:46.237735 (+ 3189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.237746 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.237870 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.237963 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:46.241129 (+ 3166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.242244 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.245028 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.245528 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.245685 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.251815 (+ 6130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:46.251825 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:46.251831 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:46.251897 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:46.256134 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.257532 (+ 1398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.257539 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.257542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.257760 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.258538 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.258547 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.258552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.259037 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:46.259218 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.260357 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.260648 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:05:46.262123 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.233403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.131:47381)
0807 18:05:46.234316 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.234387 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.237264 (+ 2877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.238666 (+ 1402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.238682 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.239660 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:46.240788 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.240819 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.242458 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.242463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.244843 (+ 2380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.244859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.245857 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.245867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.246292 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.247356 (+ 1064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:46.247410 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:46.251428 (+ 4018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:46.256329 (+ 4901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:46.256346 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:46.256370 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:46.257315 (+ 945us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:46.257936 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:46.257945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:46.257954 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:46.258351 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:46.259213 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:46.259228 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:46.259787 (+ 559us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.260218 (+ 431us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.260877 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":335,"threads_started":1}
I20260807 18:05:46.285997 10090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.268006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.8.62.131:51739)
0807 18:05:46.268883 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.268917 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.269113 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.269546 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.269554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.270340 (+ 786us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:46.270853 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.270871 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.272014 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.272022 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.274556 (+ 2534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.274595 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.275601 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.275611 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.275896 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.276704 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:46.276741 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:46.278310 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:46.282106 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:46.282115 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:46.282122 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:46.282926 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:46.283470 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:46.283478 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:46.283482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:46.283710 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:46.284620 (+ 910us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:46.284626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:46.284901 (+ 275us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.285077 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.285545 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":436,"threads_started":1}
I20260807 18:05:46.286448 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.268041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:51739 (local address 127.8.62.190:41681)
0807 18:05:46.268282 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.268291 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.269011 (+ 720us) server_negotiation.cc:407] Connection header received
0807 18:05:46.269274 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.269282 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.269381 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.269468 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:46.271051 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.271846 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.274775 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.275320 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.275479 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.278616 (+ 3137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:46.278625 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:46.278631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:46.278678 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:46.281925 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.283142 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.283148 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.283151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.283320 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.283899 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.283903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.283905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.284330 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:46.284499 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.285585 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.285914 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:05:46.298921 10004 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:39865
I20260807 18:05:46.299010 10143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:39865 every 8 connection(s)
I20260807 18:05:46.300935 10004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:05:46.303644 10144 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41681
I20260807 18:05:46.303979 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10004
I20260807 18:05:46.304165 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:05:46.304533 10144 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:46.304754 8442 external_mini_cluster.cc:1428] Setting key 5afc24411be633a9b623282a37803ec2
I20260807 18:05:46.305657 10144 heartbeater.cc:511] Master 127.8.62.190:41681 requested a full tablet report, sending...
I20260807 18:05:46.307458 9636 ts_manager.cc:194] Registered new tserver with Master: 1dffa52f291040939ceac4498abbedb5 (127.8.62.131:39865)
I20260807 18:05:46.308820 9636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:47381
I20260807 18:05:46.308956 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:05:46 dist-test-slave-rnrw krb5kdc[9589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:05:46 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:46.360502 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.345235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.8.62.190:41681)
0807 18:05:46.345497 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.345504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.345854 (+ 350us) server_negotiation.cc:407] Connection header received
0807 18:05:46.346093 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.346099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.346178 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.346298 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:46.347866 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.348725 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.350074 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.350495 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.350640 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.354846 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:46.354852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:46.354855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:46.354896 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:46.357545 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.358218 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.358224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.358229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.358367 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:46.358783 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:46.358791 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:46.358795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:46.359265 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:46.359547 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.359853 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.360103 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:05:46.365128 9636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:46.372597 9636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:46.403071 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.387500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.8.62.130:36317)
0807 18:05:46.394289 (+ 6789us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.394297 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.394333 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:05:46.394448 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.394467 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.394826 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.395939 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:46.396162 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.397746 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.400370 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.401878 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.402036 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.402278 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.402575 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.402753 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6607,"thread_start_us":6755,"threads_started":1}
I20260807 18:05:46.403160 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.386803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:36317 (local address 127.0.0.1:58856)
0807 18:05:46.388222 (+ 1419us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.388274 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.388457 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.395127 (+ 6670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.395136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.395192 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:46.395681 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.395696 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.397905 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.397912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.400198 (+ 2286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.400212 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.402103 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.402110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.402331 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.402374 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.402526 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.402738 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1119,"mutex_wait_us":26,"spinlock_wait_cycles":25472,"thread_start_us":7877,"threads_started":1}
I20260807 18:05:46.404341 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.387439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:39755 (local address 127.0.0.1:58224)
0807 18:05:46.395047 (+ 7608us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.395067 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.395188 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.396228 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.396233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.396264 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:46.396949 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.396964 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.398681 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.398686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.401287 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.401310 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.403440 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.403446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.403611 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.403672 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.403747 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.403891 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7398,"mutex_wait_us":125,"spinlock_wait_cycles":896,"thread_start_us":728,"threads_started":1}
I20260807 18:05:46.404721 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.386961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:39865 (local address 127.0.0.1:37994)
0807 18:05:46.394298 (+ 7337us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.394319 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.394448 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.396180 (+ 1732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.396186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.396219 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:46.396942 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.396957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.399269 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.399280 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.401674 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.401688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.403808 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.403814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.403977 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.404023 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.404086 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.404229 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7196,"mutex_wait_us":37,"thread_start_us":7028,"threads_started":1}
I20260807 18:05:46.405612 10161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.388030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58224 (local address 127.8.62.129:39755)
0807 18:05:46.393835 (+ 5805us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.393847 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.395330 (+ 1483us) server_negotiation.cc:407] Connection header received
0807 18:05:46.395453 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.395461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.395721 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.395936 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:46.397200 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.398533 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.401482 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.402997 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.403140 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.405044 (+ 1904us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.405128 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.405312 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5644,"thread_start_us":5529,"threads_started":1}
I20260807 18:05:46.405859 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.388199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37994 (local address 127.8.62.131:39865)
0807 18:05:46.395553 (+ 7354us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.395560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.395589 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:05:46.395727 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.395733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.395932 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.397187 (+ 1255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:46.397504 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.399115 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.401868 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.403577 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.405033 (+ 1456us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.405280 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.405357 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.405518 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7198,"thread_start_us":6976,"threads_started":1}
I20260807 18:05:46.407605 9927 tablet_service.cc:1467] Processing CreateTablet for tablet e0f8c25b66b548049622ee31a1c5c086 (DEFAULT_TABLE table=test-table [id=fc39981e10564050adf2c574f584e360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.409060 9927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f8c25b66b548049622ee31a1c5c086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.409660 9781 tablet_service.cc:1467] Processing CreateTablet for tablet e0f8c25b66b548049622ee31a1c5c086 (DEFAULT_TABLE table=test-table [id=fc39981e10564050adf2c574f584e360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.410156 10071 tablet_service.cc:1467] Processing CreateTablet for tablet e0f8c25b66b548049622ee31a1c5c086 (DEFAULT_TABLE table=test-table [id=fc39981e10564050adf2c574f584e360]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.411217 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f8c25b66b548049622ee31a1c5c086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.411736 10071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f8c25b66b548049622ee31a1c5c086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.422187 10163 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Bootstrap starting.
I20260807 18:05:46.425670 10163 tablet_bootstrap.cc:654] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.427031 10163 log.cc:824] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:46.427968 10164 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Bootstrap starting.
I20260807 18:05:46.428471 10165 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Bootstrap starting.
I20260807 18:05:46.433962 10165 tablet_bootstrap.cc:654] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.435837 10165 log.cc:824] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:46.436347 10164 tablet_bootstrap.cc:654] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.437686 10163 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: No bootstrap required, opened a new log
I20260807 18:05:46.438122 10163 ts_tablet_manager.cc:1397] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20260807 18:05:46.438824 10164 log.cc:824] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:46.440207 10165 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: No bootstrap required, opened a new log
I20260807 18:05:46.440688 10165 ts_tablet_manager.cc:1397] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260807 18:05:46.442413 10164 tablet_bootstrap.cc:492] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: No bootstrap required, opened a new log
I20260807 18:05:46.442869 10164 ts_tablet_manager.cc:1397] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260807 18:05:46.451391 10164 raft_consensus.cc:348] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.451853 10164 raft_consensus.cc:374] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.451958 10164 raft_consensus.cc:729] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443e22b7d43341bbbc24961a118b0c0c, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.452286 10163 raft_consensus.cc:348] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.452735 10164 consensus_queue.cc:260] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.453053 10163 raft_consensus.cc:374] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.453285 10163 raft_consensus.cc:729] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c989575ee4f51a81e9a00115c6820, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.454313 10163 consensus_queue.cc:260] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.455746 10165 raft_consensus.cc:348] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.456394 10165 raft_consensus.cc:374] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.456562 10165 raft_consensus.cc:729] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.457499 10164 ts_tablet_manager.cc:1428] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Time spent starting tablet: real 0.014s user 0.007s sys 0.009s
I20260807 18:05:46.457581 10165 consensus_queue.cc:260] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.459316 10000 heartbeater.cc:503] Master 127.8.62.190:41681 was elected leader, sending a full tablet report...
I20260807 18:05:46.460467 10163 ts_tablet_manager.cc:1428] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Time spent starting tablet: real 0.022s user 0.016s sys 0.006s
I20260807 18:05:46.462318 10144 heartbeater.cc:503] Master 127.8.62.190:41681 was elected leader, sending a full tablet report...
I20260807 18:05:46.463485 10165 ts_tablet_manager.cc:1428] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Time spent starting tablet: real 0.023s user 0.012s sys 0.009s
W20260807 18:05:46.465420 9855 tablet.cc:2369] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:05:46.501607 10001 tablet.cc:2369] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:05:46.553040 10145 tablet.cc:2369] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:46.681452 10171 raft_consensus.cc:482] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:46.681955 10171 raft_consensus.cc:504] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.684353 10171 leader_election.cc:288] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317), 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755)
I20260807 18:05:46.692464 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.685049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:36317 (local address 127.8.62.131:60577)
0807 18:05:46.685353 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.685375 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.685576 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.686198 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.686206 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.686235 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:46.686597 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.686607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.688494 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.688498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.690120 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.690133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.691508 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.691514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.691739 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.691781 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.691898 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.692172 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":2}
I20260807 18:05:46.692477 10160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.684841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:60577 (local address 127.8.62.130:36317)
0807 18:05:46.685074 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.685078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.685537 (+ 459us) server_negotiation.cc:407] Connection header received
0807 18:05:46.685842 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.685857 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.685998 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.686156 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:46.686816 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.688260 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.690327 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.691332 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.691468 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.691523 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.691940 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.692143 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:05:46.692883 10161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.685500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:59095 (local address 127.8.62.129:39755)
0807 18:05:46.685778 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.685782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.686663 (+ 881us) server_negotiation.cc:407] Connection header received
0807 18:05:46.686915 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.686921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.686999 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.687099 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:46.687649 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.688872 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.690839 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.691803 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.691966 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.692034 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.692415 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.692613 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:05:46.693336 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.685286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:39755 (local address 127.8.62.131:59095)
0807 18:05:46.686546 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.686566 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.686676 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.687131 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.687135 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.687160 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:46.687499 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.687508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.689017 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.689023 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.690636 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.690662 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.692027 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.692033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.692235 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.692269 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:05:46.692370 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:05:46.692934 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"mutex_wait_us":23,"thread_start_us":806,"threads_started":1}
I20260807 18:05:46.693944 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f8c25b66b548049622ee31a1c5c086" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820" is_pre_election: true
I20260807 18:05:46.694593 9947 raft_consensus.cc:2454] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:46.694530 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f8c25b66b548049622ee31a1c5c086" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c" is_pre_election: true
I20260807 18:05:46.695226 9801 raft_consensus.cc:2454] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:46.695693 10023 leader_election.cc:302] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:46.696295 10171 raft_consensus.cc:2781] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:46.696587 10171 raft_consensus.cc:482] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:46.696791 10171 raft_consensus.cc:3037] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.699950 10171 raft_consensus.cc:504] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.700742 10171 leader_election.cc:288] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Requested vote from peers ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317), 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755)
I20260807 18:05:46.701289 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f8c25b66b548049622ee31a1c5c086" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820"
I20260807 18:05:46.701522 9947 raft_consensus.cc:3037] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.701502 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f8c25b66b548049622ee31a1c5c086" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c"
I20260807 18:05:46.701776 9801 raft_consensus.cc:3037] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.704263 9947 raft_consensus.cc:2454] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:46.704952 10023 leader_election.cc:302] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:46.705322 10171 raft_consensus.cc:2781] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:46.705646 9801 raft_consensus.cc:2454] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:46.706243 10171 raft_consensus.cc:686] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 1 LEADER]: Becoming Leader. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Running, Role: LEADER
I20260807 18:05:46.706844 10171 consensus_queue.cc:237] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } }
I20260807 18:05:46.713071 9636 catalog_manager.cc:5495] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dffa52f291040939ceac4498abbedb5 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } health_report { overall_health: UNKNOWN } } }
I20260807 18:05:46.789816 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.782019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38010 (local address 127.8.62.131:39865)
0807 18:05:46.782236 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.782243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.782536 (+ 293us) server_negotiation.cc:407] Connection header received
0807 18:05:46.783045 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.783051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.783172 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.783335 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:46.783936 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.785121 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.787310 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.787776 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.787885 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.788505 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:46.789264 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:46.789388 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:46.789528 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:05:46.789602 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:05:46.799396 9947 raft_consensus.cc:1261] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:46.799607 9801 raft_consensus.cc:1261] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:46.800581 10173 consensus_queue.cc:1035] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:46.801048 10171 consensus_queue.cc:1035] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:46.813086 10178 mvcc.cc:205] Tried to move back new op lower bound from 7315971878079676416 to 7315971877715996672. Current Snapshot: MvccSnapshot[applied={T|T < 7315971878079676416}]
I20260807 18:05:46.836135 9636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:46.836608 9636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:46.842149 9636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Sending DeleteTablet for 3 replicas of tablet e0f8c25b66b548049622ee31a1c5c086
I20260807 18:05:46.843564 10071 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f8c25b66b548049622ee31a1c5c086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:37994
I20260807 18:05:46.843706 9927 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f8c25b66b548049622ee31a1c5c086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58856
I20260807 18:05:46.844801 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f8c25b66b548049622ee31a1c5c086 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58224
I20260807 18:05:46.846089 9636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:46.846994 9636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:46.849223 10188 tablet_replica.cc:330] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: stopping tablet replica
I20260807 18:05:46.849829 10188 raft_consensus.cc:2229] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:46.850487 10188 raft_consensus.cc:2258] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:46.851508 10187 tablet_replica.cc:330] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: stopping tablet replica
I20260807 18:05:46.851883 10186 tablet_replica.cc:330] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: stopping tablet replica
I20260807 18:05:46.852074 10187 raft_consensus.cc:2229] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:46.852577 10186 raft_consensus.cc:2229] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:46.852936 10187 raft_consensus.cc:2258] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:46.853775 10186 raft_consensus.cc:2258] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:46.856017 10186 ts_tablet_manager.cc:1905] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:46.857213 9781 tablet_service.cc:1467] Processing CreateTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 (DEFAULT_TABLE table=test-table [id=92da2562529b460a9cf7d15835ebc69c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.857472 9927 tablet_service.cc:1467] Processing CreateTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 (DEFAULT_TABLE table=test-table [id=92da2562529b460a9cf7d15835ebc69c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.857717 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46b38dbd15547b7acaa61711dcb1d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.858028 9927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46b38dbd15547b7acaa61711dcb1d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.866326 10186 ts_tablet_manager.cc:1918] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:05:46.866425 10071 tablet_service.cc:1467] Processing CreateTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 (DEFAULT_TABLE table=test-table [id=92da2562529b460a9cf7d15835ebc69c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:46.866882 10186 log.cc:1196] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/e0f8c25b66b548049622ee31a1c5c086
I20260807 18:05:46.867413 10163 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Bootstrap starting.
I20260807 18:05:46.868685 10187 ts_tablet_manager.cc:1905] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:46.869735 10163 tablet_bootstrap.cc:654] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.869998 10164 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Bootstrap starting.
I20260807 18:05:46.869446 10188 ts_tablet_manager.cc:1905] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:46.871488 10071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a46b38dbd15547b7acaa61711dcb1d42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:46.872036 10164 tablet_bootstrap.cc:654] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.876008 10186 ts_tablet_manager.cc:1939] T e0f8c25b66b548049622ee31a1c5c086 P 1dffa52f291040939ceac4498abbedb5: Deleting consensus metadata
I20260807 18:05:46.878501 10187 ts_tablet_manager.cc:1918] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:05:46.878764 10187 log.cc:1196] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/e0f8c25b66b548049622ee31a1c5c086
I20260807 18:05:46.878502 9621 catalog_manager.cc:4841] TS 1dffa52f291040939ceac4498abbedb5 (127.8.62.131:39865): tablet e0f8c25b66b548049622ee31a1c5c086 (table test-table [id=fc39981e10564050adf2c574f584e360]) successfully deleted
I20260807 18:05:46.879630 10187 ts_tablet_manager.cc:1939] T e0f8c25b66b548049622ee31a1c5c086 P ee1c989575ee4f51a81e9a00115c6820: Deleting consensus metadata
I20260807 18:05:46.881294 10164 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: No bootstrap required, opened a new log
I20260807 18:05:46.881511 10164 ts_tablet_manager.cc:1397] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20260807 18:05:46.882020 9622 catalog_manager.cc:4841] TS ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317): tablet e0f8c25b66b548049622ee31a1c5c086 (table test-table [id=fc39981e10564050adf2c574f584e360]) successfully deleted
I20260807 18:05:46.882694 10164 raft_consensus.cc:348] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.882989 10164 raft_consensus.cc:374] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.883279 10164 raft_consensus.cc:729] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443e22b7d43341bbbc24961a118b0c0c, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.884887 10164 consensus_queue.cc:260] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.885582 10164 ts_tablet_manager.cc:1428] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:05:46.886080 10188 ts_tablet_manager.cc:1918] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:05:46.886324 10188 log.cc:1196] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/e0f8c25b66b548049622ee31a1c5c086
I20260807 18:05:46.886976 10165 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Bootstrap starting.
I20260807 18:05:46.887104 10188 ts_tablet_manager.cc:1939] T e0f8c25b66b548049622ee31a1c5c086 P 443e22b7d43341bbbc24961a118b0c0c: Deleting consensus metadata
I20260807 18:05:46.889660 10165 tablet_bootstrap.cc:654] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:46.890679 10163 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: No bootstrap required, opened a new log
I20260807 18:05:46.890941 10163 ts_tablet_manager.cc:1397] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.006s
I20260807 18:05:46.890918 9620 catalog_manager.cc:4841] TS 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755): tablet e0f8c25b66b548049622ee31a1c5c086 (table test-table [id=fc39981e10564050adf2c574f584e360]) successfully deleted
I20260807 18:05:46.892395 10163 raft_consensus.cc:348] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.892992 10163 raft_consensus.cc:374] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.893225 10163 raft_consensus.cc:729] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c989575ee4f51a81e9a00115c6820, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.893762 10163 consensus_queue.cc:260] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.894626 10165 tablet_bootstrap.cc:492] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: No bootstrap required, opened a new log
I20260807 18:05:46.894838 10163 ts_tablet_manager.cc:1428] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:05:46.894966 10165 ts_tablet_manager.cc:1397] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260807 18:05:46.897120 10165 raft_consensus.cc:348] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.897477 10165 raft_consensus.cc:374] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:46.897749 10165 raft_consensus.cc:729] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Initialized, Role: FOLLOWER
I20260807 18:05:46.898273 10165 consensus_queue.cc:260] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.899585 10165 ts_tablet_manager.cc:1428] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 18:05:46.931674 10171 raft_consensus.cc:482] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:46.931924 10171 raft_consensus.cc:504] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.932842 10171 leader_election.cc:288] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755), ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317)
I20260807 18:05:46.933584 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46b38dbd15547b7acaa61711dcb1d42" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820" is_pre_election: true
I20260807 18:05:46.933588 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46b38dbd15547b7acaa61711dcb1d42" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c" is_pre_election: true
I20260807 18:05:46.933871 9947 raft_consensus.cc:2454] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:46.933893 9801 raft_consensus.cc:2454] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:46.934432 10023 leader_election.cc:302] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:46.934713 10171 raft_consensus.cc:2781] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:46.934854 10171 raft_consensus.cc:482] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:46.935002 10171 raft_consensus.cc:3037] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.938153 10171 raft_consensus.cc:504] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.939275 10171 leader_election.cc:288] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Requested vote from peers 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755), ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317)
I20260807 18:05:46.939487 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46b38dbd15547b7acaa61711dcb1d42" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820"
I20260807 18:05:46.939513 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a46b38dbd15547b7acaa61711dcb1d42" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c"
I20260807 18:05:46.939874 9801 raft_consensus.cc:3037] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.939893 9947 raft_consensus.cc:3037] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:46.942461 9947 raft_consensus.cc:2454] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:46.942996 10023 leader_election.cc:302] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:46.943327 10171 raft_consensus.cc:2781] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:46.943508 10171 raft_consensus.cc:686] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 1 LEADER]: Becoming Leader. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Running, Role: LEADER
I20260807 18:05:46.943626 9801 raft_consensus.cc:2454] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:46.943810 10171 consensus_queue.cc:237] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:46.947039 9637 catalog_manager.cc:5495] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dffa52f291040939ceac4498abbedb5 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } health_report { overall_health: HEALTHY } } }
Aug 07 18:05:46 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125944, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:47.008895 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.983556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41681 (local address 127.0.0.1:42594)
0807 18:05:46.984931 (+ 1375us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:46.984961 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:46.985130 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:46.985504 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:46.985509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:46.986443 (+ 934us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:46.986938 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.986962 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.988206 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.988212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.990374 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:46.990391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.991173 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.991179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:46.991372 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:46.998544 (+ 7172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:46.998576 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:47.002116 (+ 3540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:47.005964 (+ 3848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:47.005982 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:47.005991 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:47.006441 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:47.006844 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:47.006850 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:47.006853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:47.007012 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:47.007579 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:47.007588 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:47.007780 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 18:05:47.007950 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:05:47.008279 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1229,"thread_start_us":1077,"threads_started":1}
I20260807 18:05:47.008919 10198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:46.983582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.8.62.190:41681)
0807 18:05:46.984795 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:46.984801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:46.985061 (+ 260us) server_negotiation.cc:407] Connection header received
0807 18:05:46.985267 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:46.985271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:46.985330 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:46.985417 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:46.987117 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.988058 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.990539 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:46.990998 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:46.991099 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.002420 (+ 11321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.002425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.002429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.002466 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.005811 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.006579 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.006585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.006589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.006728 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.007128 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.007140 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.007143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.007438 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.007601 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.008055 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.008427 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":1029,"thread_start_us":773,"threads_started":1}
I20260807 18:05:47.014945 9636 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42594:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:05:47.016142 9636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:47.024297 9781 tablet_service.cc:1467] Processing CreateTablet for tablet b9d24b33d2044fe89f6b5086bccd330e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cadf4f9d4d8742c39d98a4ee9d417d4d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:47.024372 10071 tablet_service.cc:1467] Processing CreateTablet for tablet b9d24b33d2044fe89f6b5086bccd330e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cadf4f9d4d8742c39d98a4ee9d417d4d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:47.025051 9781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9d24b33d2044fe89f6b5086bccd330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:47.025110 10071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9d24b33d2044fe89f6b5086bccd330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:47.024912 9927 tablet_service.cc:1467] Processing CreateTablet for tablet b9d24b33d2044fe89f6b5086bccd330e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cadf4f9d4d8742c39d98a4ee9d417d4d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:47.025616 9927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9d24b33d2044fe89f6b5086bccd330e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:47.033970 10164 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: Bootstrap starting.
I20260807 18:05:47.034379 10165 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: Bootstrap starting.
I20260807 18:05:47.035806 10163 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: Bootstrap starting.
I20260807 18:05:47.036284 10164 tablet_bootstrap.cc:654] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:47.036461 10165 tablet_bootstrap.cc:654] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:47.037662 10163 tablet_bootstrap.cc:654] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:47.047501 10165 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: No bootstrap required, opened a new log
I20260807 18:05:47.047770 10165 ts_tablet_manager.cc:1397] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260807 18:05:47.049070 10165 raft_consensus.cc:348] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.049374 10165 raft_consensus.cc:374] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:47.049582 10165 raft_consensus.cc:729] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Initialized, Role: FOLLOWER
I20260807 18:05:47.049989 10165 consensus_queue.cc:260] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.050663 10171 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.051062 10164 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: No bootstrap required, opened a new log
I20260807 18:05:47.051164 10171 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.051249 10164 ts_tablet_manager.cc:1397] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.007s
I20260807 18:05:47.052079 10164 raft_consensus.cc:348] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.052525 10165 ts_tablet_manager.cc:1428] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:05:47.053561 10163 tablet_bootstrap.cc:492] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: No bootstrap required, opened a new log
I20260807 18:05:47.053954 10163 ts_tablet_manager.cc:1397] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.007s
I20260807 18:05:47.054993 10163 raft_consensus.cc:348] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.055255 10163 raft_consensus.cc:374] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:47.055367 10163 raft_consensus.cc:729] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1c989575ee4f51a81e9a00115c6820, State: Initialized, Role: FOLLOWER
I20260807 18:05:47.055637 10163 consensus_queue.cc:260] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.056366 10163 ts_tablet_manager.cc:1428] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260807 18:05:47.056099 10170 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.056564 10170 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.056607 10164 raft_consensus.cc:374] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:47.056741 10164 raft_consensus.cc:729] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443e22b7d43341bbbc24961a118b0c0c, State: Initialized, Role: FOLLOWER
I20260807 18:05:47.057051 10164 consensus_queue.cc:260] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.057538 10169 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.057870 10169 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.061038 10164 ts_tablet_manager.cc:1428] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: Time spent starting tablet: real 0.010s user 0.000s sys 0.003s
I20260807 18:05:47.266846 10171 raft_consensus.cc:482] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:47.267167 10171 raft_consensus.cc:504] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.268126 10171 leader_election.cc:288] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755), ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317)
I20260807 18:05:47.268779 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9d24b33d2044fe89f6b5086bccd330e" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820" is_pre_election: true
I20260807 18:05:47.268877 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9d24b33d2044fe89f6b5086bccd330e" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c" is_pre_election: true
I20260807 18:05:47.269065 9947 raft_consensus.cc:2454] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:47.269276 9801 raft_consensus.cc:2454] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 0.
I20260807 18:05:47.269667 10023 leader_election.cc:302] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:47.270136 10171 raft_consensus.cc:2781] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:47.270361 10171 raft_consensus.cc:482] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:47.270570 10171 raft_consensus.cc:3037] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:47.274525 10171 raft_consensus.cc:504] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.275918 9801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9d24b33d2044fe89f6b5086bccd330e" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443e22b7d43341bbbc24961a118b0c0c"
I20260807 18:05:47.276016 10171 leader_election.cc:288] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Requested vote from peers 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755), ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317)
I20260807 18:05:47.276083 9947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9d24b33d2044fe89f6b5086bccd330e" candidate_uuid: "1dffa52f291040939ceac4498abbedb5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1c989575ee4f51a81e9a00115c6820"
I20260807 18:05:47.276198 9801 raft_consensus.cc:3037] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:47.276377 9947 raft_consensus.cc:3037] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:47.279613 9801 raft_consensus.cc:2454] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:47.279685 9947 raft_consensus.cc:2454] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1dffa52f291040939ceac4498abbedb5 in term 1.
I20260807 18:05:47.280167 10023 leader_election.cc:302] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1dffa52f291040939ceac4498abbedb5, ee1c989575ee4f51a81e9a00115c6820; no voters:
I20260807 18:05:47.280613 10171 raft_consensus.cc:2781] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:47.280864 10171 raft_consensus.cc:686] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [term 1 LEADER]: Becoming Leader. State: Replica: 1dffa52f291040939ceac4498abbedb5, State: Running, Role: LEADER
I20260807 18:05:47.281167 10171 consensus_queue.cc:237] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } }
I20260807 18:05:47.284488 10173 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: TxnStatusTablet state changed. Reason: New leader 1dffa52f291040939ceac4498abbedb5. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.285156 10173 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:47.286087 9637 catalog_manager.cc:5495] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1dffa52f291040939ceac4498abbedb5 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } health_report { overall_health: HEALTHY } } }
I20260807 18:05:47.288151 10204 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:05:47.288405 10204 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:05:47.351979 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.343330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38014 (local address 127.8.62.131:39865)
0807 18:05:47.345229 (+ 1899us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.345234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.345252 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:05:47.345314 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.345317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.345384 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.345478 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:47.345975 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.346870 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.349408 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.349985 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.350098 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.350954 (+ 856us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:47.351109 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:47.351210 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.351484 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.351613 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1787,"thread_start_us":1581,"threads_started":1}
I20260807 18:05:47.352020 10199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.343329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:39865 (local address 127.0.0.1:38014)
0807 18:05:47.343657 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:47.343691 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:47.343881 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:47.345571 (+ 1690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:47.345578 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:47.345596 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:47.345807 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:47.345822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.347054 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.347061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:47.349179 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:47.349204 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.350221 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.350228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:47.350568 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.350673 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:47.351237 (+ 564us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:47.351273 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:05:47.351399 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:05:47.351602 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 18:05:47.358156 9947 raft_consensus.cc:1261] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:47.358480 9801 raft_consensus.cc:1261] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:47.358848 10173 consensus_queue.cc:1035] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:47.359355 10171 consensus_queue.cc:1035] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:47.363262 10170 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: TxnStatusTablet state changed. Reason: New leader 1dffa52f291040939ceac4498abbedb5. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.363879 10170 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.370576 10169 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: TxnStatusTablet state changed. Reason: New leader 1dffa52f291040939ceac4498abbedb5. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.371124 10169 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.371030 10170 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.371307 10170 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P ee1c989575ee4f51a81e9a00115c6820: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.377655 10207 mvcc.cc:205] Tried to move back new op lower bound from 7315971880370761728 to 7315971880066478080. Current Snapshot: MvccSnapshot[applied={T|T < 7315971880370761728}]
I20260807 18:05:47.378679 10173 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.379287 10173 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:47.380703 10215 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.381332 10215 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 1dffa52f291040939ceac4498abbedb5: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:47.388146 10169 tablet_replica.cc:439] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1dffa52f291040939ceac4498abbedb5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 } } peers { permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 } } peers { permanent_uuid: "1dffa52f291040939ceac4498abbedb5" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 39865 } } }
I20260807 18:05:47.388453 10169 tablet_replica.cc:442] T b9d24b33d2044fe89f6b5086bccd330e P 443e22b7d43341bbbc24961a118b0c0c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:47.410216 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.400913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:45623 (local address 127.8.62.131:39865)
0807 18:05:47.401139 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.401146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.401997 (+ 851us) server_negotiation.cc:407] Connection header received
0807 18:05:47.402232 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.402237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.402323 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.402433 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:47.403482 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.405131 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.407410 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.408321 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.408464 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.409140 (+ 676us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:47.409297 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:47.409398 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.409546 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.409650 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":62}
I20260807 18:05:47.410460 10218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.400918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:39865 (local address 127.8.62.131:45623)
0807 18:05:47.401884 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:47.401900 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:47.402050 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:47.402488 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:47.402492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:47.402502 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:47.403209 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:47.403220 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.405374 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.405382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:47.407239 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:47.407255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.408708 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.408716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:47.408865 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.408939 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:47.409417 (+ 478us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:47.409426 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:05:47.409498 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 18:05:47.409641 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"thread_start_us":418,"threads_started":1}
I20260807 18:05:47.422029 9947 raft_consensus.cc:1261] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:47.422029 9801 raft_consensus.cc:1261] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Refusing update from remote peer 1dffa52f291040939ceac4498abbedb5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:47.422689 10171 consensus_queue.cc:1035] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1c989575ee4f51a81e9a00115c6820" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 36317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:47.423041 10215 consensus_queue.cc:1035] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443e22b7d43341bbbc24961a118b0c0c" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 39755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:47.489840 9637 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:47.490160 9637 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42582:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:47.494005 9637 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa1740bf1f0b480b9d1bc3e1b3e32034: Sending DeleteTablet for 3 replicas of tablet a46b38dbd15547b7acaa61711dcb1d42
I20260807 18:05:47.495220 9781 tablet_service.cc:1514] Processing DeleteTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:47 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58224
I20260807 18:05:47.495318 10071 tablet_service.cc:1514] Processing DeleteTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:47 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:37994
I20260807 18:05:47.495990 9927 tablet_service.cc:1514] Processing DeleteTablet for tablet a46b38dbd15547b7acaa61711dcb1d42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:47 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58856
I20260807 18:05:47.504503 10226 tablet_replica.cc:330] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: stopping tablet replica
I20260807 18:05:47.504760 10226 raft_consensus.cc:2229] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:47.506974 10226 raft_consensus.cc:2258] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:47.509660 10226 ts_tablet_manager.cc:1905] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:47.511598 10223 tablet_replica.cc:330] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: stopping tablet replica
I20260807 18:05:47.511848 10223 raft_consensus.cc:2229] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:47.512303 10223 raft_consensus.cc:2258] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:47.513335 10224 tablet_replica.cc:330] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: stopping tablet replica
I20260807 18:05:47.513571 10224 raft_consensus.cc:2229] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:47.514259 10224 raft_consensus.cc:2258] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:47.516031 10224 ts_tablet_manager.cc:1905] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:47.518699 10226 ts_tablet_manager.cc:1918] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:47.518888 10226 log.cc:1196] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/a46b38dbd15547b7acaa61711dcb1d42
I20260807 18:05:47.519762 10226 ts_tablet_manager.cc:1939] T a46b38dbd15547b7acaa61711dcb1d42 P ee1c989575ee4f51a81e9a00115c6820: Deleting consensus metadata
I20260807 18:05:47.520668 10223 ts_tablet_manager.cc:1905] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:47.521262 9622 catalog_manager.cc:4841] TS ee1c989575ee4f51a81e9a00115c6820 (127.8.62.130:36317): tablet a46b38dbd15547b7acaa61711dcb1d42 (table test-table [id=92da2562529b460a9cf7d15835ebc69c]) successfully deleted
I20260807 18:05:47.531114 10224 ts_tablet_manager.cc:1918] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:47.531327 10223 ts_tablet_manager.cc:1918] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:47.531384 10224 log.cc:1196] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/a46b38dbd15547b7acaa61711dcb1d42
I20260807 18:05:47.531507 10223 log.cc:1196] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/a46b38dbd15547b7acaa61711dcb1d42
I20260807 18:05:47.532020 10224 ts_tablet_manager.cc:1939] T a46b38dbd15547b7acaa61711dcb1d42 P 1dffa52f291040939ceac4498abbedb5: Deleting consensus metadata
I20260807 18:05:47.532109 10223 ts_tablet_manager.cc:1939] T a46b38dbd15547b7acaa61711dcb1d42 P 443e22b7d43341bbbc24961a118b0c0c: Deleting consensus metadata
I20260807 18:05:47.533595 9620 catalog_manager.cc:4841] TS 443e22b7d43341bbbc24961a118b0c0c (127.8.62.129:39755): tablet a46b38dbd15547b7acaa61711dcb1d42 (table test-table [id=92da2562529b460a9cf7d15835ebc69c]) successfully deleted
I20260807 18:05:47.533725 9621 catalog_manager.cc:4841] TS 1dffa52f291040939ceac4498abbedb5 (127.8.62.131:39865): tablet a46b38dbd15547b7acaa61711dcb1d42 (table test-table [id=92da2562529b460a9cf7d15835ebc69c]) successfully deleted
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:47.547178 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.510031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.8.62.129:39755)
0807 18:05:47.523921 (+ 13890us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.523927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.523954 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:05:47.524015 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.524019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.524160 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.524291 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:47.527117 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.529074 (+ 1957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.531092 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.532716 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.532861 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.539059 (+ 6198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.539078 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.539096 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.539181 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.543301 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.544183 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.544192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.544194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.544421 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.544937 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.544942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.544944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.545348 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.545691 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.546577 (+ 886us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.546778 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13753,"thread_start_us":13606,"threads_started":1}
W20260807 18:05:47.548215 9849 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58240
I20260807 18:05:47.580399 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.563045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.8.62.190:41681)
0807 18:05:47.563819 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.563828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.565139 (+ 1311us) server_negotiation.cc:407] Connection header received
0807 18:05:47.565339 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.565344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.565425 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.565547 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:47.567521 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.568369 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.570065 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.570575 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.570726 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.573757 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.573764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.573770 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.573812 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.577431 (+ 3619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.578203 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.578213 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.578216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.578369 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.578785 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.578789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.578791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.579080 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.579385 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.579714 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.579954 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":412,"threads_started":1}
W20260807 18:05:47.581521 9637 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42606
I20260807 18:05:47.583488 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9713
I20260807 18:05:47.607012 10228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.585021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.8.62.129:39755)
0807 18:05:47.585281 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.585289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.592782 (+ 7493us) server_negotiation.cc:407] Connection header received
0807 18:05:47.592999 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.593003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.593081 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.593207 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:47.594712 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.595516 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.596955 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.597384 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.597530 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.600430 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.600437 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.600440 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.600491 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.603670 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.604521 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.604529 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.604533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.604748 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.605186 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.605192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.605195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.605573 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.605845 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.606214 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.606562 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 18:05:47.607961 9849 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58252
W20260807 18:05:47.608668 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9713: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:47.608899 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9713
I20260807 18:05:47.635185 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9858
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:05:47.653906 10234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.636146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58872 (local address 127.8.62.130:36317)
0807 18:05:47.636737 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.636741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.636763 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:05:47.636817 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.636821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.636882 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.636979 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:47.638709 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.639519 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.641479 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.642036 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.642145 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.647508 (+ 5363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.647526 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.647543 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.647584 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.650779 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.651636 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.651645 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.651648 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.651793 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.652206 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.652212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.652215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.652656 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.653022 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.653266 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.653532 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":313,"threads_started":1}
W20260807 18:05:47.654627 9994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58872
W20260807 18:05:47.655313 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9858: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:47.655565 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9858
I20260807 18:05:47.680401 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10004
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[9589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125946, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:05:47.698429 10206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:47.681388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.8.62.131:39865)
0807 18:05:47.681604 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:47.681609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:47.681631 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:05:47.681722 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:47.681726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:47.681806 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:47.681931 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:47.683296 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.684602 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.685881 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:47.686573 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:47.686708 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:47.690978 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:47.690997 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:47.691024 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:47.691080 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:47.695444 (+ 4364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.696167 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.696177 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.696180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.696368 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:47.696792 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:47.696798 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:47.696802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:47.697295 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:47.697648 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:47.697811 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 18:05:47.698038 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:05:47.699433 10139 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38020
W20260807 18:05:47.700170 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10004: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:47.700349 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10004
I20260807 18:05:47.725392 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 9604
I20260807 18:05:47.726637 9678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:47.842932 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 9604
2026-08-07T18:05:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5066 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[10238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[10238](info): set up 2 sockets
Aug 07 18:05:47 dist-test-slave-rnrw krb5kdc[10238](info): commencing operation
krb5kdc: starting...
W20260807 18:05:48.963461 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:05:49 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:05:49.169368 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46455
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:49.502561 10251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:49.502878 10251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:49.503041 10251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:49.510840 10251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:05:49.510977 10251 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:05:49.511062 10251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:49.511198 10251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:49.511291 10251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:05:49.511386 10251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:05:49.521799 10251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46455
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:49.523200 10251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:49.524695 10251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:49.538380 10257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:49.538439 10259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:49.538744 10256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:49.539858 10251 server_base.cc:1034] running on GCE node
I20260807 18:05:49.540750 10251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:49.542344 10251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:49.543645 10251 hybrid_clock.cc:648] HybridClock initialized: now 1786125949543575 us; error 43 us; skew 500 ppm
I20260807 18:05:49.544049 10251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:49 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:49.547619 10251 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:05:49.549373 10251 webserver.cc:460] Webserver started at http://127.8.62.190:33515/ using document root <none> and password file <none>
I20260807 18:05:49.550194 10251 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:49.550331 10251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:49.550681 10251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:49.553942 10251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "3ce1e50c05fc485eb5da6f550c49fffc"
format_stamp: "Formatted at 2026-08-07 18:05:49 on dist-test-slave-rnrw"
server_key: "5cf00da613978bb54660d00540bf278b"
server_key_iv: "5dda25e4d2d3cfb1e855754b513babdd"
server_key_version: "encryptionkey@0"
I20260807 18:05:49.554786 10251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "3ce1e50c05fc485eb5da6f550c49fffc"
format_stamp: "Formatted at 2026-08-07 18:05:49 on dist-test-slave-rnrw"
server_key: "5cf00da613978bb54660d00540bf278b"
server_key_iv: "5dda25e4d2d3cfb1e855754b513babdd"
server_key_version: "encryptionkey@0"
I20260807 18:05:49.560370 10251 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:05:49.564180 10267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:49.565758 10251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:05:49.565966 10251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "3ce1e50c05fc485eb5da6f550c49fffc"
format_stamp: "Formatted at 2026-08-07 18:05:49 on dist-test-slave-rnrw"
server_key: "5cf00da613978bb54660d00540bf278b"
server_key_iv: "5dda25e4d2d3cfb1e855754b513babdd"
server_key_version: "encryptionkey@0"
I20260807 18:05:49.566232 10251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:49.586236 10251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:49.590960 10251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:49.591261 10251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:49.613461 10251 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46455
I20260807 18:05:49.613481 10330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46455 every 8 connection(s)
I20260807 18:05:49.614902 10251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:05:49.618842 10331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:49.620959 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10251
I20260807 18:05:49.621182 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:05:49.621980 8442 external_mini_cluster.cc:1428] Setting key 76da278c39bda19f6c4afa2f6a950da1
I20260807 18:05:49.631920 10331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Bootstrap starting.
I20260807 18:05:49.635933 10331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:49.637329 10331 log.cc:824] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:05:49 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:49.640321 10331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: No bootstrap required, opened a new log
I20260807 18:05:49.646626 10334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:49.624216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.8.62.190:46455)
0807 18:05:49.625126 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:49.625139 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:49.625182 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:05:49.626887 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:49.626911 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:49.627309 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:49.627654 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:49.629528 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:49.630825 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:49.632757 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:49.633543 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:49.633703 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:49.639018 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:49.639026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:49.639041 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:49.639076 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:49.642449 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:49.643203 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:49.643210 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:49.643216 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:49.643387 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:49.643809 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:49.643812 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:49.643814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:49.644354 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:49.644723 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:49.645177 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:05:49.645592 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":440,"threads_started":1}
I20260807 18:05:49.648844 10331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } }
I20260807 18:05:49.649245 10331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:49.649372 10331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce1e50c05fc485eb5da6f550c49fffc, State: Initialized, Role: FOLLOWER
I20260807 18:05:49.650065 10331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } }
I20260807 18:05:49.650326 10331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:05:49.650470 10331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:05:49.650652 10331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:49.652895 10331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } }
I20260807 18:05:49.653396 10331 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce1e50c05fc485eb5da6f550c49fffc; no voters:
I20260807 18:05:49.654002 10331 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:05:49.654142 10336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:49.655635 10336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [term 1 LEADER]: Becoming Leader. State: Replica: 3ce1e50c05fc485eb5da6f550c49fffc, State: Running, Role: LEADER
I20260807 18:05:49.656242 10331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:05:49.656142 10336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } }
I20260807 18:05:49.659384 10338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce1e50c05fc485eb5da6f550c49fffc. Latest consensus state: current_term: 1 leader_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } } }
I20260807 18:05:49.659752 10338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:49.660154 10337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce1e50c05fc485eb5da6f550c49fffc" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46455 } } }
I20260807 18:05:49.660463 10337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:49.661537 10345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:05:49.666826 10345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:05:49.675658 10345 catalog_manager.cc:1347] Generated new cluster ID: ab10f56f85484128ba92474de61e86a2
I20260807 18:05:49.675802 10345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:05:49.697531 10345 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:05:49.698912 10345 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:05:49.712704 10345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Generated new TSK 0
I20260807 18:05:49.713640 10345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:05:49.764016 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46455
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:50.102433 10358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:50.102805 10358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:50.102978 10358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:50.110500 10358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:50.110690 10358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:50.110844 10358 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:05:50.110932 10358 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:50.111066 10358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:05:50.121130 10358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:50.122545 10358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:50.123957 10358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:50.138041 10367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:50.138206 10365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:50.138141 10363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:50.139549 10358 server_base.cc:1034] running on GCE node
I20260807 18:05:50.140264 10358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:50.141397 10358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:50.142701 10358 hybrid_clock.cc:648] HybridClock initialized: now 1786125950142617 us; error 77 us; skew 500 ppm
I20260807 18:05:50.143046 10358 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:50 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:50.146603 10358 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:05:50.148185 10358 webserver.cc:460] Webserver started at http://127.8.62.129:40141/ using document root <none> and password file <none>
I20260807 18:05:50.148980 10358 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:50.149091 10358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:50.149369 10358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:50.152266 10358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "460eabba634a429e92cda02e78d2d745"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "02dbe3313effe0ea6eb2ba967186bd38"
server_key_iv: "50c64c3e94e5cdb6fb4136a5ba83c2b8"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.153059 10358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "460eabba634a429e92cda02e78d2d745"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "02dbe3313effe0ea6eb2ba967186bd38"
server_key_iv: "50c64c3e94e5cdb6fb4136a5ba83c2b8"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.158584 10358 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:05:50.162432 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:50.163928 10358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:05:50.164083 10358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "460eabba634a429e92cda02e78d2d745"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "02dbe3313effe0ea6eb2ba967186bd38"
server_key_iv: "50c64c3e94e5cdb6fb4136a5ba83c2b8"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.164336 10358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:50.216595 10358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:50.221282 10358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:50.221585 10358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:50.224256 10358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:50.224350 10358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:50.224596 10358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:50.224712 10358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:50 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:50.251175 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.225200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:41393 (local address 127.8.62.190:46455)
0807 18:05:50.225705 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:50.225711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:50.227005 (+ 1294us) server_negotiation.cc:407] Connection header received
0807 18:05:50.230347 (+ 3342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:50.230354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:50.230459 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:50.230568 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:50.233814 (+ 3246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.235151 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.237515 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.238307 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.238466 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.242488 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:50.242495 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:50.242501 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:50.242547 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:50.246391 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.247924 (+ 1533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.247929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.247933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.248124 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.248833 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.248837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.248839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.249183 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:50.249358 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:50.250722 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0807 18:05:50.250908 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":247,"threads_started":1}
I20260807 18:05:50.253808 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.225867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.129:41393)
0807 18:05:50.226790 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:50.226854 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:50.229914 (+ 3060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:50.231281 (+ 1367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:50.231297 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:50.232632 (+ 1335us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:50.233551 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.233589 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.235372 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.235379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.237323 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.237346 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.238577 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.238582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.238766 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.239616 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:50.239645 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:50.242213 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:50.246704 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.246717 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.246741 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.247736 (+ 995us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.248252 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.248261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.248264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.248662 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.249352 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:50.249364 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:50.250130 (+ 766us) client_negotiation.cc:769] Sending connection context
0807 18:05:50.250599 (+ 469us) client_negotiation.cc:240] Negotiation successful
0807 18:05:50.251430 (+ 831us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":347,"threads_started":1}
I20260807 18:05:50.275902 10358 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:33161
I20260807 18:05:50.275991 10499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:33161 every 8 connection(s)
I20260807 18:05:50.278069 10358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:05:50.278787 10475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.262916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.129:56213)
0807 18:05:50.263620 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:50.263648 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:50.263816 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:50.264290 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:50.264295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:50.264986 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:50.265521 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.265546 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.267365 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.267377 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.269396 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.269413 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.270351 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.270358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.270535 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.271374 (+ 839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:50.271392 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:50.272158 (+ 766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:50.275616 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.275621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.275626 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.276297 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.276792 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.276795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.276798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.276938 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.277681 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:50.277684 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:50.277857 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 18:05:50.277978 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:05:50.278367 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":513,"threads_started":1}
I20260807 18:05:50.278894 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.263368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:56213 (local address 127.8.62.190:46455)
0807 18:05:50.263625 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:50.263635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:50.263728 (+ 93us) server_negotiation.cc:407] Connection header received
0807 18:05:50.263976 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:50.263996 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:50.264109 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:50.264246 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:50.265729 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.267188 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.269607 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.270142 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.270308 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.272374 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:50.272383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:50.272389 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:50.272452 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:50.275471 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.276445 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.276451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.276455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.276664 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.277070 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.277075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.277079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.277539 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:50.277717 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:50.278158 (+ 441us) server_negotiation.cc:299] Negotiation successful
0807 18:05:50.278429 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:05:50.280411 10500 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46455
I20260807 18:05:50.280730 10500 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:50.281526 10500 heartbeater.cc:511] Master 127.8.62.190:46455 requested a full tablet report, sending...
I20260807 18:05:50.283810 10285 ts_manager.cc:194] Registered new tserver with Master: 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161)
I20260807 18:05:50.285394 10285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:41393
I20260807 18:05:50.286916 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10358
I20260807 18:05:50.287156 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:05:50.287971 8442 external_mini_cluster.cc:1428] Setting key 28f1c91b14d5cac0449890bc5bac9712
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:05:50.332609 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46455
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:50.674777 10505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:50.675086 10505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:50.675254 10505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:50.682725 10505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:50.682845 10505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:50.683020 10505 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:05:50.683084 10505 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:50.683177 10505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:05:50.693363 10505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:50.694779 10505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:50.696157 10505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:50.710691 10510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:50.710630 10513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:50.710695 10511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:50.711396 10505 server_base.cc:1034] running on GCE node
I20260807 18:05:50.712105 10505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:50.713312 10505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:50.714625 10505 hybrid_clock.cc:648] HybridClock initialized: now 1786125950714580 us; error 56 us; skew 500 ppm
I20260807 18:05:50.714977 10505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:50 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:50.718406 10505 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:05:50.719848 10505 webserver.cc:460] Webserver started at http://127.8.62.130:38625/ using document root <none> and password file <none>
I20260807 18:05:50.720487 10505 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:50.720660 10505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:50.720989 10505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:50.723883 10505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "fcac58336a2946548ef501b1b0cdce14"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "29cad6b38b1a1f817e5140adfdf99a0e"
server_key_iv: "4589a98dcea13c0bcdc9e8f2921f17d6"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.724712 10505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "fcac58336a2946548ef501b1b0cdce14"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "29cad6b38b1a1f817e5140adfdf99a0e"
server_key_iv: "4589a98dcea13c0bcdc9e8f2921f17d6"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.730254 10505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:05:50.734189 10521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:50.735719 10505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:05:50.735939 10505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "fcac58336a2946548ef501b1b0cdce14"
format_stamp: "Formatted at 2026-08-07 18:05:50 on dist-test-slave-rnrw"
server_key: "29cad6b38b1a1f817e5140adfdf99a0e"
server_key_iv: "4589a98dcea13c0bcdc9e8f2921f17d6"
server_key_version: "encryptionkey@0"
I20260807 18:05:50.736177 10505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:50.760129 10505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:50.765872 10505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:50.766225 10505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:50.769675 10505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:50.769867 10505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:50.770006 10505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:50.770125 10505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:50 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125950, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:50.794983 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.770185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:35921 (local address 127.8.62.190:46455)
0807 18:05:50.771243 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:50.771249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:50.771802 (+ 553us) server_negotiation.cc:407] Connection header received
0807 18:05:50.773976 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:50.773984 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:50.774085 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:50.774205 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:50.776472 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.777792 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.780425 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.781154 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.781289 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.786787 (+ 5498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:50.786793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:50.786798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:50.786848 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:50.790784 (+ 3936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.792201 (+ 1417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.792208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.792214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.792403 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.793092 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.793098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.793104 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.793432 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:50.793599 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:50.794377 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 18:05:50.794610 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879}
I20260807 18:05:50.796095 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.770709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.130:35921)
0807 18:05:50.771624 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:50.771684 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:50.773696 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:50.774449 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:50.774457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:50.775239 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:50.776223 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.776257 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.778013 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.778020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.780248 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.780265 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.781523 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.781534 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.781853 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.782740 (+ 887us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:50.782782 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:50.786550 (+ 3768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:50.791030 (+ 4480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.791051 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.791080 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.792039 (+ 959us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.792578 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.792583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.792587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.792906 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.793582 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:50.793591 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:50.793981 (+ 390us) client_negotiation.cc:769] Sending connection context
0807 18:05:50.794267 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:05:50.794827 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":458,"threads_started":1}
I20260807 18:05:50.818281 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.802396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:40499 (local address 127.8.62.190:46455)
0807 18:05:50.802556 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:50.802560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:50.803479 (+ 919us) server_negotiation.cc:407] Connection header received
0807 18:05:50.803717 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:50.803721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:50.803782 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:50.803859 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:50.805648 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.806625 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.808826 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.809439 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.809573 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.812058 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:50.812066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:50.812074 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:50.812125 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:50.815410 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.816076 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.816081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.816085 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.816244 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:50.816647 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:50.816654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:50.816659 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:50.817072 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:50.817255 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:50.817864 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 18:05:50.818027 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 18:05:50.819010 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:50.802390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.130:40499)
0807 18:05:50.803342 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:50.803371 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:50.803572 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:50.803946 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:50.803954 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:50.804819 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:50.805457 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.805476 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.806777 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.806782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.808637 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:50.808667 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:50.809656 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:50.809665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:50.809829 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:50.810578 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:50.810596 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:50.811847 (+ 1251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:50.815542 (+ 3695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.815549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.815553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.815950 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.816360 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:50.816365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:50.816369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:50.816503 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:50.817257 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:50.817264 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:50.817561 (+ 297us) client_negotiation.cc:769] Sending connection context
0807 18:05:50.817774 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 18:05:50.818275 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":382,"threads_started":1}
I20260807 18:05:50.836450 10505 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:46323
I20260807 18:05:50.836472 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:46323 every 8 connection(s)
I20260807 18:05:50.837874 10505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:05:50.840157 10647 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46455
I20260807 18:05:50.840361 10647 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:50.841055 10647 heartbeater.cc:511] Master 127.8.62.190:46455 requested a full tablet report, sending...
I20260807 18:05:50.842458 10285 ts_manager.cc:194] Registered new tserver with Master: fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323)
I20260807 18:05:50.843247 10285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:35921
I20260807 18:05:50.846127 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10505
I20260807 18:05:50.846310 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:05:50.846709 8442 external_mini_cluster.cc:1428] Setting key 03e0fc99a13035ab547b6a87d7d3b024
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:05:50.893983 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46455
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:51.241914 10651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:51.242261 10651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:51.242446 10651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:51.249789 10651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:51.249967 10651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:51.250130 10651 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 18:05:51.250221 10651 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:05:51.250337 10651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:05:51.260713 10651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:46455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:51.262111 10651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:51.263533 10651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:51.279542 10656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:51.279778 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:51.280169 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:51.280763 10651 server_base.cc:1034] running on GCE node
I20260807 18:05:51.281517 10651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:51.282611 10651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:51.283874 10651 hybrid_clock.cc:648] HybridClock initialized: now 1786125951283805 us; error 66 us; skew 500 ppm
I20260807 18:05:51.284248 10651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:51 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125951, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:51.287771 10651 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:05:51.288399 10500 heartbeater.cc:503] Master 127.8.62.190:46455 was elected leader, sending a full tablet report...
I20260807 18:05:51.289798 10651 webserver.cc:460] Webserver started at http://127.8.62.131:43759/ using document root <none> and password file <none>
I20260807 18:05:51.290719 10651 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:51.290863 10651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:51.291218 10651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:51.294410 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "0d72067fa003487cb20e580452a3279e"
format_stamp: "Formatted at 2026-08-07 18:05:51 on dist-test-slave-rnrw"
server_key: "deca9121be7819bd00f48fabd5d663fa"
server_key_iv: "1aa4a8e17935916d83491e135889d83f"
server_key_version: "encryptionkey@0"
I20260807 18:05:51.295338 10651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "0d72067fa003487cb20e580452a3279e"
format_stamp: "Formatted at 2026-08-07 18:05:51 on dist-test-slave-rnrw"
server_key: "deca9121be7819bd00f48fabd5d663fa"
server_key_iv: "1aa4a8e17935916d83491e135889d83f"
server_key_version: "encryptionkey@0"
I20260807 18:05:51.301468 10651 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:05:51.305655 10666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:51.307372 10651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:05:51.307559 10651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "0d72067fa003487cb20e580452a3279e"
format_stamp: "Formatted at 2026-08-07 18:05:51 on dist-test-slave-rnrw"
server_key: "deca9121be7819bd00f48fabd5d663fa"
server_key_iv: "1aa4a8e17935916d83491e135889d83f"
server_key_version: "encryptionkey@0"
I20260807 18:05:51.307823 10651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:51.350005 10651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:51.354540 10651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:51.354866 10651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:51.358474 10651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:51.358608 10651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:51.358734 10651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:51.358830 10651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:05:51 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125951, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:51.390377 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.359303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44295 (local address 127.8.62.190:46455)
0807 18:05:51.368240 (+ 8937us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.368249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.368274 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:05:51.368371 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.368376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.368494 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.368758 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:51.371178 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.372052 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.374668 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.375343 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.375505 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.381146 (+ 5641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:51.381154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:51.381162 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:51.381222 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:51.385851 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.387115 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.387119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.387121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.387266 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.387972 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.387977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.387979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.388366 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:51.388504 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.389862 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.390085 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8786,"thread_start_us":8659,"threads_started":1}
I20260807 18:05:51.392583 10676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.359920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.131:44295)
0807 18:05:51.361160 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.361241 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.365012 (+ 3771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.369117 (+ 4105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.369137 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.370325 (+ 1188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:51.370975 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.371004 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.372238 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.372245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.374400 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.374425 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.375619 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.375630 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.375887 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.376877 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:51.376921 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:51.380775 (+ 3854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:51.386060 (+ 5285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.386078 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.386104 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.386915 (+ 811us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.387382 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.387390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.387397 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.387809 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.388515 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:51.388525 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:51.389267 (+ 742us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.389704 (+ 437us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.390526 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":495,"threads_started":1}
I20260807 18:05:51.415056 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.399408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:52161 (local address 127.8.62.190:46455)
0807 18:05:51.399612 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.399617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.400326 (+ 709us) server_negotiation.cc:407] Connection header received
0807 18:05:51.400607 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.400613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.400733 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.400849 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:51.402164 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.403236 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.404608 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.405163 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.405304 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.408282 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:51.408287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:51.408290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:51.408324 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:51.411770 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.412831 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.412839 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.412842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.412997 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.413425 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.413431 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.413434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.413840 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:51.414011 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.414454 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.414689 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:05:51.415179 10744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.399536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.8.62.131:52161)
0807 18:05:51.400208 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.400228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.400403 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.400894 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.400902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.401472 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:51.402008 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.402020 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.403374 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.403382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.404432 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.404444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.405589 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.405599 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.405859 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.406620 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:51.406668 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:51.408013 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:51.411954 (+ 3941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.411958 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.411961 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.412678 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.413128 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.413139 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.413141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.413304 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.413970 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:51.413974 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:51.414174 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.414321 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.414688 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":371,"threads_started":1}
I20260807 18:05:51.422238 10651 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:38689
I20260807 18:05:51.422259 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:38689 every 8 connection(s)
I20260807 18:05:51.423647 10651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:05:51.426110 10793 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46455
I20260807 18:05:51.426352 10793 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:51.427122 10793 heartbeater.cc:511] Master 127.8.62.190:46455 requested a full tablet report, sending...
I20260807 18:05:51.428867 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10651
I20260807 18:05:51.428871 10285 ts_manager.cc:194] Registered new tserver with Master: 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:51.429056 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:05:51.429502 8442 external_mini_cluster.cc:1428] Setting key f4e0bb0b945233972adea581fffc49d0
I20260807 18:05:51.429914 10285 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:44295
I20260807 18:05:51.432885 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:05:51 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:05:51 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:51.485066 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.469329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.8.62.190:46455)
0807 18:05:51.469535 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.469542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.470019 (+ 477us) server_negotiation.cc:407] Connection header received
0807 18:05:51.470246 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.470251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.470335 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.470529 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:51.472089 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.472924 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.474260 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.474684 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.474791 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.478976 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:51.478983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:51.478986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:51.479024 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:51.481835 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.482671 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.482675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.482677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.482826 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.483263 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.483267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.483269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.483613 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:51.483816 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.484282 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.484614 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:05:51.489825 10285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:51.497081 10285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:51.531865 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.512039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53368 (local address 127.8.62.130:46323)
0807 18:05:51.513570 (+ 1531us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.513586 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.520410 (+ 6824us) server_negotiation.cc:407] Connection header received
0807 18:05:51.520640 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.520646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.520892 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.521106 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:51.522652 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.524664 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.528664 (+ 4000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.530072 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.530204 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.530405 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.531329 (+ 924us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.531529 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216,"thread_start_us":3182,"threads_started":1}
I20260807 18:05:51.532655 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.511817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46323 (local address 127.0.0.1:53368)
0807 18:05:51.517166 (+ 5349us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.517192 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.517306 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.521947 (+ 4641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.521954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.521987 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:51.522420 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.522433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.524877 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.524886 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.528332 (+ 3446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.528356 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.530540 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.530547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.530792 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.530865 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.531036 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.532178 (+ 1142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5211,"mutex_wait_us":1020,"thread_start_us":3872,"threads_started":1}
I20260807 18:05:51.533313 10811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.517603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34120 (local address 127.8.62.129:33161)
0807 18:05:51.520404 (+ 2801us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.520411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.520457 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:05:51.520592 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.520600 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.520892 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.521106 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:51.521928 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.524689 (+ 2761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.529565 (+ 4876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.531274 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.531453 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.531704 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.532770 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.532961 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2646,"thread_start_us":2500,"threads_started":1}
I20260807 18:05:51.534386 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.517288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:33161 (local address 127.0.0.1:34120)
0807 18:05:51.517885 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.517904 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.518024 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.521179 (+ 3155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.521190 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.521235 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:51.521732 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.521749 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.524924 (+ 3175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.524930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.529341 (+ 4411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.529356 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.531871 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.531882 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.532063 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.532093 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.532177 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.532345 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":127,"thread_start_us":300,"threads_started":1}
I20260807 18:05:51.537724 10579 tablet_service.cc:1467] Processing CreateTablet for tablet bdc4f53148294735afd37105bba7aa14 (DEFAULT_TABLE table=test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:51.539070 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.511521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.0.0.1:60512)
0807 18:05:51.516124 (+ 4603us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.516175 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.516387 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.528212 (+ 11825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.528217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.528246 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:51.528751 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.528765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.530792 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.530797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.532891 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.532907 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.537844 (+ 4937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.537851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.538042 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.538080 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.538218 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.538429 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4288,"mutex_wait_us":27,"thread_start_us":5349,"threads_started":1}
I20260807 18:05:51.538759 10426 tablet_service.cc:1467] Processing CreateTablet for tablet bdc4f53148294735afd37105bba7aa14 (DEFAULT_TABLE table=test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:51.539592 10579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc4f53148294735afd37105bba7aa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.540489 10426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc4f53148294735afd37105bba7aa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.541375 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.511988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.8.62.131:38689)
0807 18:05:51.527563 (+ 15575us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.527572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.527608 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:05:51.527723 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.527731 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.528013 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.528238 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:51.528987 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.530623 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.533126 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.537575 (+ 4449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.539423 (+ 1848us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.539969 (+ 546us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.540140 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.540463 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15354,"thread_start_us":16503,"threads_started":1}
I20260807 18:05:51.545380 10717 tablet_service.cc:1467] Processing CreateTablet for tablet bdc4f53148294735afd37105bba7aa14 (DEFAULT_TABLE table=test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:51.546933 10717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdc4f53148294735afd37105bba7aa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.566681 10814 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Bootstrap starting.
I20260807 18:05:51.566900 10812 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Bootstrap starting.
I20260807 18:05:51.570629 10813 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Bootstrap starting.
I20260807 18:05:51.571543 10814 tablet_bootstrap.cc:654] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.573143 10814 log.cc:824] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:51.575690 10813 tablet_bootstrap.cc:654] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.576170 10814 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: No bootstrap required, opened a new log
I20260807 18:05:51.576449 10814 ts_tablet_manager.cc:1397] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:05:51.576996 10813 log.cc:824] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:51.577844 10812 tablet_bootstrap.cc:654] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.579210 10812 log.cc:824] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:51.580367 10813 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: No bootstrap required, opened a new log
I20260807 18:05:51.580729 10813 ts_tablet_manager.cc:1397] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:05:51.582510 10812 tablet_bootstrap.cc:492] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: No bootstrap required, opened a new log
I20260807 18:05:51.582787 10812 ts_tablet_manager.cc:1397] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260807 18:05:51.585177 10814 raft_consensus.cc:348] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.585558 10814 raft_consensus.cc:374] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.585687 10814 raft_consensus.cc:729] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.586437 10814 consensus_queue.cc:260] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.587666 10793 heartbeater.cc:503] Master 127.8.62.190:46455 was elected leader, sending a full tablet report...
I20260807 18:05:51.588357 10814 ts_tablet_manager.cc:1428] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260807 18:05:51.591090 10813 raft_consensus.cc:348] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.591490 10813 raft_consensus.cc:374] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.591619 10813 raft_consensus.cc:729] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.592315 10813 consensus_queue.cc:260] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.594267 10813 ts_tablet_manager.cc:1428] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260807 18:05:51.594285 10812 raft_consensus.cc:348] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.594798 10812 raft_consensus.cc:374] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.594918 10812 raft_consensus.cc:729] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.595561 10812 consensus_queue.cc:260] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.596719 10647 heartbeater.cc:503] Master 127.8.62.190:46455 was elected leader, sending a full tablet report...
I20260807 18:05:51.597294 10812 ts_tablet_manager.cc:1428] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260807 18:05:51.659691 10820 raft_consensus.cc:482] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:51.660120 10820 raft_consensus.cc:504] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.662119 10820 leader_election.cc:288] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689), fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323)
I20260807 18:05:51.672808 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.662723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:39195 (local address 127.8.62.131:38689)
0807 18:05:51.662949 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.662956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.663824 (+ 868us) server_negotiation.cc:407] Connection header received
0807 18:05:51.664063 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.664068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.664152 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.664263 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:51.665316 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.666643 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.670177 (+ 3534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.671594 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.671751 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.671833 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.672347 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.672510 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:05:51.673080 10823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.662454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.8.62.129:39195)
0807 18:05:51.663639 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.663665 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.663898 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.664336 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.664342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.664383 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:51.665033 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.665043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.666928 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.666934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.669947 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.669967 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.671791 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.671796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.672107 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.672185 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.672294 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.672479 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":403,"spinlock_wait_cycles":25472,"thread_start_us":755,"threads_started":1}
I20260807 18:05:51.673257 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.662733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:35245 (local address 127.8.62.130:46323)
0807 18:05:51.662949 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.662956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.663830 (+ 874us) server_negotiation.cc:407] Connection header received
0807 18:05:51.664062 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.664068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.664171 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.664261 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:51.665311 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.666680 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.670177 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.671451 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.671676 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.671765 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.672812 (+ 1047us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.672962 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:05:51.673295 10822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.662486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46323 (local address 127.8.62.129:35245)
0807 18:05:51.663708 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.663726 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.663898 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.664336 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.664342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.664383 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:51.665004 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.665025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.666927 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.666934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.669947 (+ 3013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.669967 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.671720 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.671730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.672114 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.672188 (+ 74us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.672294 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.672479 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":309,"spinlock_wait_cycles":28544,"thread_start_us":713,"threads_started":1}
I20260807 18:05:51.674391 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc4f53148294735afd37105bba7aa14" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14" is_pre_election: true
I20260807 18:05:51.674988 10599 raft_consensus.cc:2454] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 0.
I20260807 18:05:51.675236 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc4f53148294735afd37105bba7aa14" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e" is_pre_election: true
W20260807 18:05:51.675652 10794 tablet.cc:2369] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:51.675889 10742 raft_consensus.cc:2454] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 0.
I20260807 18:05:51.675952 10378 leader_election.cc:302] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460eabba634a429e92cda02e78d2d745, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:51.676425 10820 raft_consensus.cc:2781] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:51.676658 10820 raft_consensus.cc:482] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:51.676805 10820 raft_consensus.cc:3037] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:51.679026 10820 raft_consensus.cc:504] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.679652 10820 leader_election.cc:288] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 election: Requested vote from peers 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689), fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323)
I20260807 18:05:51.680107 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc4f53148294735afd37105bba7aa14" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e"
I20260807 18:05:51.680131 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdc4f53148294735afd37105bba7aa14" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14"
I20260807 18:05:51.680330 10742 raft_consensus.cc:3037] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:51.680330 10599 raft_consensus.cc:3037] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:51.683636 10742 raft_consensus.cc:2454] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 1.
I20260807 18:05:51.683636 10599 raft_consensus.cc:2454] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 1.
I20260807 18:05:51.684113 10377 leader_election.cc:302] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, 460eabba634a429e92cda02e78d2d745; no voters:
I20260807 18:05:51.684435 10820 raft_consensus.cc:2781] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:51.684942 10820 raft_consensus.cc:686] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 1 LEADER]: Becoming Leader. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Running, Role: LEADER
I20260807 18:05:51.685442 10820 consensus_queue.cc:237] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } }
I20260807 18:05:51.690511 10284 catalog_manager.cc:5495] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 460eabba634a429e92cda02e78d2d745 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "460eabba634a429e92cda02e78d2d745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } health_report { overall_health: HEALTHY } } }
Aug 07 18:05:51 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125949, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:51.761242 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.747313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46455 (local address 127.0.0.1:51330)
0807 18:05:51.748048 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:51.748066 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:51.748205 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:51.748541 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:51.748603 (+ 62us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:51.749324 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:51.749624 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.749635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.750655 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.750659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.751631 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:51.751642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.752381 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.752385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:51.752496 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.753271 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:51.753288 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:51.755464 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:51.758619 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.758630 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.758637 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.759157 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.759497 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:51.759501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:51.759503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:51.759642 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:51.760221 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:51.760227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:51.760374 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 18:05:51.760683 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 18:05:51.760902 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":410,"threads_started":1}
I20260807 18:05:51.761322 10675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:51.747244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.8.62.190:46455)
0807 18:05:51.747469 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:51.747478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:51.748178 (+ 700us) server_negotiation.cc:407] Connection header received
0807 18:05:51.748309 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:51.748313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:51.748374 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:51.748461 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:51.749762 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.750511 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.751758 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:51.752225 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:51.752319 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:51.755636 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:51.755642 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:51.755645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:51.755683 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:51.758457 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.759264 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.759268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.759270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.759387 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:51.759738 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:51.759741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:51.759742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:51.760050 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:51.760208 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:51.760790 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:05:51.761021 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":67}
I20260807 18:05:51.766775 10284 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:51330:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:05:51.767922 10284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:51.775130 10426 tablet_service.cc:1467] Processing CreateTablet for tablet 8ddd1a79412f43e28de9a4afffa33f87 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5043aaede77844019cb7a4a061aa567f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:51.775167 10579 tablet_service.cc:1467] Processing CreateTablet for tablet 8ddd1a79412f43e28de9a4afffa33f87 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5043aaede77844019cb7a4a061aa567f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:51.776197 10579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ddd1a79412f43e28de9a4afffa33f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.776324 10426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ddd1a79412f43e28de9a4afffa33f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.776511 10717 tablet_service.cc:1467] Processing CreateTablet for tablet 8ddd1a79412f43e28de9a4afffa33f87 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5043aaede77844019cb7a4a061aa567f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:05:51.777359 10717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ddd1a79412f43e28de9a4afffa33f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:51.782670 10814 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: Bootstrap starting.
I20260807 18:05:51.783665 10813 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: Bootstrap starting.
I20260807 18:05:51.784379 10814 tablet_bootstrap.cc:654] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.785413 10813 tablet_bootstrap.cc:654] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.785580 10812 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: Bootstrap starting.
W20260807 18:05:51.785693 10501 tablet.cc:2369] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:51.787361 10812 tablet_bootstrap.cc:654] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:51.787959 10813 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: No bootstrap required, opened a new log
I20260807 18:05:51.788122 10813 ts_tablet_manager.cc:1397] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:05:51.789013 10813 raft_consensus.cc:348] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.789489 10813 raft_consensus.cc:374] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.789556 10814 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: No bootstrap required, opened a new log
I20260807 18:05:51.789609 10813 raft_consensus.cc:729] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.789721 10814 ts_tablet_manager.cc:1397] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:05:51.789887 10813 consensus_queue.cc:260] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.790208 10812 tablet_bootstrap.cc:492] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: No bootstrap required, opened a new log
I20260807 18:05:51.790427 10812 ts_tablet_manager.cc:1397] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:05:51.790477 10820 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:51.790539 10814 raft_consensus.cc:348] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.790809 10820 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:51.790808 10814 raft_consensus.cc:374] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.790902 10814 raft_consensus.cc:729] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.791150 10814 consensus_queue.cc:260] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.791447 10812 raft_consensus.cc:348] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.791682 10812 raft_consensus.cc:374] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:51.791780 10814 ts_tablet_manager.cc:1428] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:05:51.791772 10812 raft_consensus.cc:729] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Initialized, Role: FOLLOWER
I20260807 18:05:51.791612 10819 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:51.791983 10819 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:51.792002 10812 consensus_queue.cc:260] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:51.792668 10812 ts_tablet_manager.cc:1428] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 18:05:51.792536 10821 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:51.793025 10821 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:51.794178 10813 ts_tablet_manager.cc:1428] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
W20260807 18:05:51.840313 10648 tablet.cc:2369] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:05:52.126034 10819 raft_consensus.cc:482] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:52.126338 10819 raft_consensus.cc:504] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.128141 10819 leader_election.cc:288] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323)
I20260807 18:05:52.134773 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.128531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:36829 (local address 127.8.62.129:33161)
0807 18:05:52.129292 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.129301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.129323 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:05:52.129400 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.129404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.129465 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.129590 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:52.130111 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.130925 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.132804 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.133697 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.133893 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.133992 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.134276 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.134435 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":604,"threads_started":1}
I20260807 18:05:52.134896 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.128875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:45899 (local address 127.8.62.130:46323)
0807 18:05:52.129135 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.129143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.129778 (+ 635us) server_negotiation.cc:407] Connection header received
0807 18:05:52.130018 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.130023 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.130092 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.130199 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:52.130685 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.131429 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.133075 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.133880 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.134077 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.134163 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.134466 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.134609 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 18:05:52.135115 10836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.128397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:33161 (local address 127.8.62.131:36829)
0807 18:05:52.128986 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.129012 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.129189 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.129611 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.129616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.129655 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:52.129883 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.129893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.131077 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.131086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.132606 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.132629 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.133882 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.133889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.134092 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.134123 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.134249 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.134412 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":5,"thread_start_us":988,"threads_started":1}
I20260807 18:05:52.135340 10835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.128409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46323 (local address 127.8.62.131:45899)
0807 18:05:52.129652 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.129668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.129846 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.130223 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.130227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.130253 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:52.130509 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.130518 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.131584 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.131590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.132905 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.132917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.134121 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.134128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.134300 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.134324 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.134436 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.134568 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1136,"mutex_wait_us":5,"thread_start_us":309,"threads_started":1}
I20260807 18:05:52.136054 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ddd1a79412f43e28de9a4afffa33f87" candidate_uuid: "0d72067fa003487cb20e580452a3279e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14" is_pre_election: true
I20260807 18:05:52.136389 10599 raft_consensus.cc:2454] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d72067fa003487cb20e580452a3279e in term 0.
I20260807 18:05:52.136250 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ddd1a79412f43e28de9a4afffa33f87" candidate_uuid: "0d72067fa003487cb20e580452a3279e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745" is_pre_election: true
I20260807 18:05:52.136723 10448 raft_consensus.cc:2454] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d72067fa003487cb20e580452a3279e in term 0.
I20260807 18:05:52.136971 10670 leader_election.cc:302] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:52.137557 10819 raft_consensus.cc:2781] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:52.137689 10819 raft_consensus.cc:482] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:52.137818 10819 raft_consensus.cc:3037] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.140420 10819 raft_consensus.cc:504] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.141337 10819 leader_election.cc:288] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [CANDIDATE]: Term 1 election: Requested vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323)
I20260807 18:05:52.141853 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ddd1a79412f43e28de9a4afffa33f87" candidate_uuid: "0d72067fa003487cb20e580452a3279e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745"
I20260807 18:05:52.141973 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ddd1a79412f43e28de9a4afffa33f87" candidate_uuid: "0d72067fa003487cb20e580452a3279e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14"
I20260807 18:05:52.142100 10448 raft_consensus.cc:3037] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.142233 10599 raft_consensus.cc:3037] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.143571 10820 consensus_queue.cc:1035] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.145025 10599 raft_consensus.cc:2454] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d72067fa003487cb20e580452a3279e in term 1.
I20260807 18:05:52.145506 10448 raft_consensus.cc:2454] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d72067fa003487cb20e580452a3279e in term 1.
I20260807 18:05:52.145534 10670 leader_election.cc:302] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:52.146536 10838 raft_consensus.cc:2781] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:52.146821 10838 raft_consensus.cc:686] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [term 1 LEADER]: Becoming Leader. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Running, Role: LEADER
I20260807 18:05:52.147435 10838 consensus_queue.cc:237] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.150540 10819 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: TxnStatusTablet state changed. Reason: New leader 0d72067fa003487cb20e580452a3279e. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.150827 10819 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:52.154894 10284 catalog_manager.cc:5495] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d72067fa003487cb20e580452a3279e (127.8.62.131). New cstate: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } health_report { overall_health: HEALTHY } } }
I20260807 18:05:52.156250 10842 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:05:52.156679 10842 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:05:52.156843 10820 consensus_queue.cc:1035] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.257094 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.250939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.8.62.131:38689)
0807 18:05:52.251553 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.251561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.251576 (+ 15us) server_negotiation.cc:407] Connection header received
0807 18:05:52.251666 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.251670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.251747 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.251918 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.252382 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.253297 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.254593 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.255030 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.255170 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.255631 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.256257 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.256412 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.256666 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.256736 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":300,"threads_started":1}
I20260807 18:05:52.257138 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.250807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.0.0.1:60518)
0807 18:05:52.251051 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.251090 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.251271 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.251938 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.251943 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.251956 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:52.252210 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.252228 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.253436 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.253444 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.254439 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.254451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.255188 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.255194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.255364 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.255422 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.256387 (+ 965us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.256415 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.256524 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.256717 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:05:52.264302 10599 raft_consensus.cc:1261] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Refusing update from remote peer 0d72067fa003487cb20e580452a3279e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:52.264477 10448 raft_consensus.cc:1261] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Refusing update from remote peer 0d72067fa003487cb20e580452a3279e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:52.265311 10838 consensus_queue.cc:1035] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.265650 10819 consensus_queue.cc:1035] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e [LEADER]: Connected to new peer: Peer: permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.273716 10821 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: TxnStatusTablet state changed. Reason: New leader 0d72067fa003487cb20e580452a3279e. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.273984 10821 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:52.276006 10843 mvcc.cc:205] Tried to move back new op lower bound from 7315971900466274304 to 7315971900001230848. Current Snapshot: MvccSnapshot[applied={T|T < 7315971900466274304}]
I20260807 18:05:52.276876 10838 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.277192 10838 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:52.277590 10846 mvcc.cc:205] Tried to move back new op lower bound from 7315971900466274304 to 7315971900001230848. Current Snapshot: MvccSnapshot[applied={T|T < 7315971900466274304}]
I20260807 18:05:52.281620 10821 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.281875 10821 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P fcac58336a2946548ef501b1b0cdce14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:52.291168 10820 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: TxnStatusTablet state changed. Reason: New leader 0d72067fa003487cb20e580452a3279e. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.291486 10820 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:52.291713 10819 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.291997 10819 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 0d72067fa003487cb20e580452a3279e: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:05:52.294180 10841 mvcc.cc:205] Tried to move back new op lower bound from 7315971900466274304 to 7315971900001230848. Current Snapshot: MvccSnapshot[applied={T|T < 7315971900466274304}]
I20260807 18:05:52.296726 10820 tablet_replica.cc:439] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0d72067fa003487cb20e580452a3279e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } } }
I20260807 18:05:52.297016 10820 tablet_replica.cc:442] T 8ddd1a79412f43e28de9a4afffa33f87 P 460eabba634a429e92cda02e78d2d745: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:05:52.306805 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.298909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.8.62.129:33161)
0807 18:05:52.299081 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.299087 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.300633 (+ 1546us) server_negotiation.cc:407] Connection header received
0807 18:05:52.300968 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.300975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.301077 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.301244 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.301661 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.302815 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.304266 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.304939 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.305077 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.305482 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.306208 (+ 726us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.306357 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.306415 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.306518 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:05:52.325186 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.318564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:52835 (local address 127.8.62.131:38689)
0807 18:05:52.318803 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.318810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.319461 (+ 651us) server_negotiation.cc:407] Connection header received
0807 18:05:52.319676 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.319681 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.319754 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.319837 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.320376 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.321196 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.323290 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.323728 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.323859 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.324451 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.324642 (+ 191us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.324726 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.324894 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.324964 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:05:52.325433 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.318316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.8.62.129:52835)
0807 18:05:52.319335 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.319363 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.319533 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.319884 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.319889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.319902 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:52.320205 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.320218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.321347 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.321353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.323130 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.323147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.323919 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.323926 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.324130 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.324232 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.324755 (+ 523us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.324768 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.324863 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.325037 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":676,"threads_started":1}
I20260807 18:05:52.371150 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.363500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:54729 (local address 127.8.62.129:33161)
0807 18:05:52.363690 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.363698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.363849 (+ 151us) server_negotiation.cc:407] Connection header received
0807 18:05:52.364005 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.364012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.364141 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.364244 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.364849 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.366023 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.368292 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.368970 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.369134 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.370171 (+ 1037us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.370408 (+ 237us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.370507 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.370721 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.370804 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:05:52.371265 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.363088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:33161 (local address 127.8.62.131:54729)
0807 18:05:52.363751 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.363772 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.363887 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.364267 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.364270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.364280 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:52.364674 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.364686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.366215 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.366221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.368079 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.368096 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.369663 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.369674 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.369862 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.369949 (+ 87us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.370569 (+ 620us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.370583 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.370692 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.370849 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":397,"threads_started":1}
I20260807 18:05:52.416970 10285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:52.417305 10285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:52.421795 10285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Sending DeleteTablet for 3 replicas of tablet bdc4f53148294735afd37105bba7aa14
I20260807 18:05:52.423170 10579 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc4f53148294735afd37105bba7aa14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:52 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:53368
I20260807 18:05:52.423337 10717 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc4f53148294735afd37105bba7aa14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:52 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:60512
I20260807 18:05:52.423377 10426 tablet_service.cc:1514] Processing DeleteTablet for tablet bdc4f53148294735afd37105bba7aa14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:52 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34120
I20260807 18:05:52.427532 10285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:52.428328 10285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:52.437637 10425 tablet_service.cc:1467] Processing CreateTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 (DEFAULT_TABLE table=test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:52.438349 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83cd46d392f4d869c2fa80eeda87d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:52.441139 10578 tablet_service.cc:1467] Processing CreateTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 (DEFAULT_TABLE table=test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:52.441741 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83cd46d392f4d869c2fa80eeda87d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:52.442113 10867 tablet_replica.cc:330] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: stopping tablet replica
I20260807 18:05:52.443832 10867 raft_consensus.cc:2229] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:52.444033 10868 tablet_replica.cc:330] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: stopping tablet replica
I20260807 18:05:52.444470 10868 raft_consensus.cc:2229] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:52.444953 10867 raft_consensus.cc:2258] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:52.445695 10868 raft_consensus.cc:2258] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:52.448463 10716 tablet_service.cc:1467] Processing CreateTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 (DEFAULT_TABLE table=test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:52.449153 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c83cd46d392f4d869c2fa80eeda87d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:52.449304 10868 ts_tablet_manager.cc:1905] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:52.461086 10870 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Bootstrap starting.
I20260807 18:05:52.461707 10871 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Bootstrap starting.
I20260807 18:05:52.462842 10867 ts_tablet_manager.cc:1905] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:52.463104 10870 tablet_bootstrap.cc:654] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:52.465250 10868 ts_tablet_manager.cc:1918] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:52.465476 10868 log.cc:1196] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/bdc4f53148294735afd37105bba7aa14
I20260807 18:05:52.466614 10871 tablet_bootstrap.cc:654] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:52.466812 10868 ts_tablet_manager.cc:1939] T bdc4f53148294735afd37105bba7aa14 P 460eabba634a429e92cda02e78d2d745: Deleting consensus metadata
I20260807 18:05:52.471256 10270 catalog_manager.cc:4841] TS 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161): tablet bdc4f53148294735afd37105bba7aa14 (table test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]) successfully deleted
I20260807 18:05:52.472389 10869 tablet_replica.cc:330] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: stopping tablet replica
I20260807 18:05:52.472920 10869 raft_consensus.cc:2229] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:52.473340 10869 raft_consensus.cc:2258] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:52.473836 10870 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: No bootstrap required, opened a new log
I20260807 18:05:52.474004 10870 ts_tablet_manager.cc:1397] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.000s
I20260807 18:05:52.474483 10874 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Bootstrap starting.
I20260807 18:05:52.474843 10870 raft_consensus.cc:348] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.475121 10870 raft_consensus.cc:374] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:52.475235 10870 raft_consensus.cc:729] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Initialized, Role: FOLLOWER
I20260807 18:05:52.475587 10870 consensus_queue.cc:260] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.477952 10874 tablet_bootstrap.cc:654] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:52.478899 10870 ts_tablet_manager.cc:1428] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 18:05:52.479979 10871 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: No bootstrap required, opened a new log
I20260807 18:05:52.480180 10871 ts_tablet_manager.cc:1397] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260807 18:05:52.480177 10867 ts_tablet_manager.cc:1918] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:52.480371 10867 log.cc:1196] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/bdc4f53148294735afd37105bba7aa14
I20260807 18:05:52.480994 10867 ts_tablet_manager.cc:1939] T bdc4f53148294735afd37105bba7aa14 P fcac58336a2946548ef501b1b0cdce14: Deleting consensus metadata
I20260807 18:05:52.481194 10871 raft_consensus.cc:348] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.481518 10871 raft_consensus.cc:374] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:52.481671 10871 raft_consensus.cc:729] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Initialized, Role: FOLLOWER
I20260807 18:05:52.481987 10871 consensus_queue.cc:260] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.482784 10271 catalog_manager.cc:4841] TS fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323): tablet bdc4f53148294735afd37105bba7aa14 (table test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]) successfully deleted
I20260807 18:05:52.483698 10871 ts_tablet_manager.cc:1428] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:05:52.479018 10869 ts_tablet_manager.cc:1905] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:52.493974 10874 tablet_bootstrap.cc:492] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: No bootstrap required, opened a new log
I20260807 18:05:52.494184 10874 ts_tablet_manager.cc:1397] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.000s
I20260807 18:05:52.494351 10869 ts_tablet_manager.cc:1918] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:52.494622 10869 log.cc:1196] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/bdc4f53148294735afd37105bba7aa14
I20260807 18:05:52.495306 10869 ts_tablet_manager.cc:1939] T bdc4f53148294735afd37105bba7aa14 P 0d72067fa003487cb20e580452a3279e: Deleting consensus metadata
I20260807 18:05:52.495219 10874 raft_consensus.cc:348] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.495508 10874 raft_consensus.cc:374] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:52.495635 10874 raft_consensus.cc:729] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Initialized, Role: FOLLOWER
I20260807 18:05:52.495903 10874 consensus_queue.cc:260] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.496587 10874 ts_tablet_manager.cc:1428] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:05:52.498080 10270 catalog_manager.cc:4841] TS 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689): tablet bdc4f53148294735afd37105bba7aa14 (table test-table [id=e8b6096f7ad44409b0c69f1ff6a6ea95]) successfully deleted
I20260807 18:05:52.752801 10821 raft_consensus.cc:482] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:52.753103 10821 raft_consensus.cc:504] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.754679 10821 leader_election.cc:288] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:52.762863 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.755078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45413 (local address 127.8.62.129:33161)
0807 18:05:52.755333 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.755339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.755666 (+ 327us) server_negotiation.cc:407] Connection header received
0807 18:05:52.755906 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.755915 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.756037 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.756128 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:52.757097 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.758177 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.760826 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.761732 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.761919 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.762011 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.762309 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.762445 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:05:52.762974 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.754898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:33161 (local address 127.8.62.130:45413)
0807 18:05:52.755532 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.755553 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.755710 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.756406 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.756414 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.756449 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:52.756837 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.756854 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.758336 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.758342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.760637 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.760651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.761920 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.761926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.762115 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.762141 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.762249 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.762392 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":2,"thread_start_us":399,"threads_started":1}
I20260807 18:05:52.764156 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83cd46d392f4d869c2fa80eeda87d60" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745" is_pre_election: true
I20260807 18:05:52.764613 10448 raft_consensus.cc:2454] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 0.
I20260807 18:05:52.765471 10524 leader_election.cc:302] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460eabba634a429e92cda02e78d2d745, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:52.766219 10821 raft_consensus.cc:2781] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:52.766383 10821 raft_consensus.cc:482] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:52.766510 10821 raft_consensus.cc:3037] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.766739 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.755774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.8.62.130:55569)
0807 18:05:52.757371 (+ 1597us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.757393 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.757647 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.758096 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.758101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.758131 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:05:52.759000 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.759020 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.760676 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.760682 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.763380 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.763396 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.765447 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.765458 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.765781 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.765820 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.765990 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.766133 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1432,"mutex_wait_us":27,"thread_start_us":1172,"threads_started":1}
I20260807 18:05:52.767091 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.755988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:55569 (local address 127.8.62.131:38689)
0807 18:05:52.756208 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.756214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.757568 (+ 1354us) server_negotiation.cc:407] Connection header received
0807 18:05:52.757847 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.757853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.757944 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.758052 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:05:52.759394 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.760441 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.763569 (+ 3128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.765110 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.765292 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.765382 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.765989 (+ 607us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.766133 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:05:52.769253 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83cd46d392f4d869c2fa80eeda87d60" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e" is_pre_election: true
I20260807 18:05:52.769568 10742 raft_consensus.cc:2454] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 0.
I20260807 18:05:52.770773 10821 raft_consensus.cc:504] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.771658 10821 leader_election.cc:288] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 election: Requested vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:52.772296 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83cd46d392f4d869c2fa80eeda87d60" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745"
I20260807 18:05:52.772358 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c83cd46d392f4d869c2fa80eeda87d60" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e"
I20260807 18:05:52.772634 10448 raft_consensus.cc:3037] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.772708 10742 raft_consensus.cc:3037] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:52.775049 10742 raft_consensus.cc:2454] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 1.
I20260807 18:05:52.775516 10524 leader_election.cc:302] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:52.775818 10821 raft_consensus.cc:2781] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:52.776360 10448 raft_consensus.cc:2454] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 1.
I20260807 18:05:52.781405 10821 raft_consensus.cc:686] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 1 LEADER]: Becoming Leader. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Running, Role: LEADER
I20260807 18:05:52.781914 10821 consensus_queue.cc:237] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:52.787853 10285 catalog_manager.cc:5495] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcac58336a2946548ef501b1b0cdce14 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "fcac58336a2946548ef501b1b0cdce14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260807 18:05:52.841032 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.823144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53370 (local address 127.8.62.130:46323)
0807 18:05:52.824115 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.824122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.832699 (+ 8577us) server_negotiation.cc:407] Connection header received
0807 18:05:52.832936 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.832940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.833033 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.833225 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.834022 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.835169 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.837756 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.838477 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.838607 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.839332 (+ 725us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.840205 (+ 873us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.840385 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.840498 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.840623 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":784,"thread_start_us":636,"threads_started":1}
I20260807 18:05:52.865080 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.856385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:33999 (local address 127.8.62.131:38689)
0807 18:05:52.856603 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.856611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.857236 (+ 625us) server_negotiation.cc:407] Connection header received
0807 18:05:52.857522 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.857528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.857630 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.857849 (+ 219us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.858997 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.860221 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.862576 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.863348 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.863546 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.864109 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.864283 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.864432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.864572 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.864666 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 18:05:52.865381 10885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.856287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:38689 (local address 127.8.62.130:33999)
0807 18:05:52.857098 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.857119 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.857352 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.857895 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.857901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.857914 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:52.858743 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.858756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.860365 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.860371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.862400 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.862416 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.863609 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.863616 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.863813 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.863891 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.864414 (+ 523us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.864426 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.864505 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.864707 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":366,"threads_started":1}
I20260807 18:05:52.876940 10742 raft_consensus.cc:1261] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Refusing update from remote peer fcac58336a2946548ef501b1b0cdce14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:52.877261 10448 raft_consensus.cc:1261] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Refusing update from remote peer fcac58336a2946548ef501b1b0cdce14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:52.877704 10878 consensus_queue.cc:1035] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.878331 10821 consensus_queue.cc:1035] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:52.921432 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.914371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46323 (local address 127.8.62.131:57649)
0807 18:05:52.915416 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:52.915438 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:52.915604 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:52.916023 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:52.916028 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:52.916045 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:05:52.916307 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.916321 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.917550 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.917555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.919139 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:52.919152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.920301 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.920307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:52.920441 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.920480 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.920967 (+ 487us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.920976 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:05:52.921039 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 18:05:52.921158 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":604,"threads_started":1}
I20260807 18:05:52.921542 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:52.914692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:57649 (local address 127.8.62.130:46323)
0807 18:05:52.915033 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:52.915041 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:52.915530 (+ 489us) server_negotiation.cc:407] Connection header received
0807 18:05:52.915824 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:52.915828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:52.915894 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:52.915963 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:52.916502 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.917356 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.919309 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:52.920015 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:52.920166 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:52.920713 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:52.920855 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:52.920960 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:52.921067 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 18:05:52.921159 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
W20260807 18:05:52.922894 10846 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:05:54.546548 10284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:54.546780 10284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:54.551231 10284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Sending DeleteTablet for 3 replicas of tablet c83cd46d392f4d869c2fa80eeda87d60
I20260807 18:05:54.552294 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34120
I20260807 18:05:54.552330 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:60512
I20260807 18:05:54.552392 10578 tablet_service.cc:1514] Processing DeleteTablet for tablet c83cd46d392f4d869c2fa80eeda87d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:53368
I20260807 18:05:54.563726 10918 tablet_replica.cc:330] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: stopping tablet replica
I20260807 18:05:54.564330 10918 raft_consensus.cc:2229] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:54.565250 10918 raft_consensus.cc:2258] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.566830 10918 ts_tablet_manager.cc:1905] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:54.569164 10916 tablet_replica.cc:330] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: stopping tablet replica
I20260807 18:05:54.569480 10916 raft_consensus.cc:2229] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:54.569865 10916 raft_consensus.cc:2258] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.569875 10917 tablet_replica.cc:330] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: stopping tablet replica
I20260807 18:05:54.570180 10917 raft_consensus.cc:2229] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:54.570482 10917 raft_consensus.cc:2258] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.571187 10916 ts_tablet_manager.cc:1905] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:54.571632 10917 ts_tablet_manager.cc:1905] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:54.578154 10918 ts_tablet_manager.cc:1918] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:54.578428 10918 log.cc:1196] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/c83cd46d392f4d869c2fa80eeda87d60
I20260807 18:05:54.579038 10918 ts_tablet_manager.cc:1939] T c83cd46d392f4d869c2fa80eeda87d60 P fcac58336a2946548ef501b1b0cdce14: Deleting consensus metadata
I20260807 18:05:54.579272 10916 ts_tablet_manager.cc:1918] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:54.579612 10916 log.cc:1196] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/c83cd46d392f4d869c2fa80eeda87d60
I20260807 18:05:54.580338 10916 ts_tablet_manager.cc:1939] T c83cd46d392f4d869c2fa80eeda87d60 P 460eabba634a429e92cda02e78d2d745: Deleting consensus metadata
I20260807 18:05:54.580425 10917 ts_tablet_manager.cc:1918] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:54.580420 10271 catalog_manager.cc:4841] TS fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323): tablet c83cd46d392f4d869c2fa80eeda87d60 (table test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]) successfully deleted
I20260807 18:05:54.580621 10917 log.cc:1196] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/c83cd46d392f4d869c2fa80eeda87d60
I20260807 18:05:54.581023 10917 ts_tablet_manager.cc:1939] T c83cd46d392f4d869c2fa80eeda87d60 P 0d72067fa003487cb20e580452a3279e: Deleting consensus metadata
I20260807 18:05:54.581789 10270 catalog_manager.cc:4841] TS 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161): tablet c83cd46d392f4d869c2fa80eeda87d60 (table test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]) successfully deleted
I20260807 18:05:54.582163 10270 catalog_manager.cc:4841] TS 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689): tablet c83cd46d392f4d869c2fa80eeda87d60 (table test-table [id=e35c0e31bb7c4c12982e46a5aeac2541]) successfully deleted
Aug 07 18:05:54 dist-test-slave-rnrw krb5kdc[10238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:05:54 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:54.626902 10926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:54.603934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.8.62.190:46455)
0807 18:05:54.604730 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:54.604735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:54.604759 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:05:54.604909 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:54.604913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:54.604985 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:54.605083 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:54.606618 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:54.607528 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:54.609170 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:54.609622 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:54.609744 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:54.620092 (+ 10348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:54.620105 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:54.620109 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:54.620156 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:54.623815 (+ 3659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:54.624655 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:54.624660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:54.624662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:54.624856 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:54.625307 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:54.625312 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:54.625314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:54.625640 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:54.625926 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:54.626267 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:05:54.626514 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":372,"threads_started":1}
I20260807 18:05:54.630802 10284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:05:54.631438 10284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:54.638514 10425 tablet_service.cc:1467] Processing CreateTablet for tablet deab871748734a2fa9585e96390f8646 (DEFAULT_TABLE table=test-table [id=ae2570fd97974032a0926a5d6ace6f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.638695 10578 tablet_service.cc:1467] Processing CreateTablet for tablet deab871748734a2fa9585e96390f8646 (DEFAULT_TABLE table=test-table [id=ae2570fd97974032a0926a5d6ace6f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.639070 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab871748734a2fa9585e96390f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.638984 10716 tablet_service.cc:1467] Processing CreateTablet for tablet deab871748734a2fa9585e96390f8646 (DEFAULT_TABLE table=test-table [id=ae2570fd97974032a0926a5d6ace6f7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.639209 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab871748734a2fa9585e96390f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.639447 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet deab871748734a2fa9585e96390f8646. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.645869 10927 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Bootstrap starting.
I20260807 18:05:54.647058 10928 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Bootstrap starting.
I20260807 18:05:54.648036 10927 tablet_bootstrap.cc:654] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.648523 10929 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Bootstrap starting.
I20260807 18:05:54.649009 10928 tablet_bootstrap.cc:654] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.651265 10929 tablet_bootstrap.cc:654] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.652349 10928 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: No bootstrap required, opened a new log
I20260807 18:05:54.652702 10928 ts_tablet_manager.cc:1397] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 18:05:54.653563 10927 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: No bootstrap required, opened a new log
I20260807 18:05:54.653729 10927 ts_tablet_manager.cc:1397] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260807 18:05:54.654105 10928 raft_consensus.cc:348] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.654471 10928 raft_consensus.cc:374] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.654598 10928 raft_consensus.cc:729] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.654564 10927 raft_consensus.cc:348] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.654826 10927 raft_consensus.cc:374] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.654953 10927 raft_consensus.cc:729] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.654866 10928 consensus_queue.cc:260] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.655257 10927 consensus_queue.cc:260] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.655815 10928 ts_tablet_manager.cc:1428] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:05:54.655560 10929 tablet_bootstrap.cc:492] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: No bootstrap required, opened a new log
I20260807 18:05:54.656417 10929 ts_tablet_manager.cc:1397] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260807 18:05:54.657203 10927 ts_tablet_manager.cc:1428] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 18:05:54.657581 10929 raft_consensus.cc:348] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.657850 10929 raft_consensus.cc:374] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.657976 10929 raft_consensus.cc:729] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.658269 10929 consensus_queue.cc:260] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.659577 10929 ts_tablet_manager.cc:1428] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 18:05:54.814786 10933 raft_consensus.cc:482] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:54.815207 10933 raft_consensus.cc:504] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.816221 10933 leader_election.cc:288] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:54.816818 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab871748734a2fa9585e96390f8646" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e" is_pre_election: true
I20260807 18:05:54.816938 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab871748734a2fa9585e96390f8646" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14" is_pre_election: true
I20260807 18:05:54.817126 10742 raft_consensus.cc:2454] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 0.
I20260807 18:05:54.817282 10599 raft_consensus.cc:2454] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 0.
I20260807 18:05:54.817699 10377 leader_election.cc:302] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, 460eabba634a429e92cda02e78d2d745; no voters:
I20260807 18:05:54.818056 10933 raft_consensus.cc:2781] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:54.818238 10933 raft_consensus.cc:482] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:54.818379 10933 raft_consensus.cc:3037] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:54.821524 10933 raft_consensus.cc:504] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.822144 10933 leader_election.cc:288] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 election: Requested vote from peers fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:54.822710 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab871748734a2fa9585e96390f8646" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e"
I20260807 18:05:54.822657 10599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "deab871748734a2fa9585e96390f8646" candidate_uuid: "460eabba634a429e92cda02e78d2d745" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcac58336a2946548ef501b1b0cdce14"
I20260807 18:05:54.823006 10742 raft_consensus.cc:3037] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:54.823060 10599 raft_consensus.cc:3037] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:54.826570 10742 raft_consensus.cc:2454] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 1.
I20260807 18:05:54.826562 10599 raft_consensus.cc:2454] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 460eabba634a429e92cda02e78d2d745 in term 1.
I20260807 18:05:54.827042 10378 leader_election.cc:302] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, 460eabba634a429e92cda02e78d2d745, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:54.827462 10933 raft_consensus.cc:2781] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:54.828022 10933 raft_consensus.cc:686] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 1 LEADER]: Becoming Leader. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Running, Role: LEADER
I20260807 18:05:54.828399 10933 consensus_queue.cc:237] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.831118 10285 catalog_manager.cc:5495] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 reported cstate change: term changed from 0 to 1, leader changed from <none> to 460eabba634a429e92cda02e78d2d745 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "460eabba634a429e92cda02e78d2d745" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260807 18:05:54.869194 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:54.862542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34148 (local address 127.8.62.129:33161)
0807 18:05:54.863477 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:54.863482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:54.863503 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:05:54.863563 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:54.863566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:54.863636 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:54.863734 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:54.864176 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:54.865450 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:54.867003 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:54.867490 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:54.867592 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:54.868278 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:54.868437 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:54.868528 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:54.868791 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:05:54.868865 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827,"thread_start_us":676,"threads_started":1}
I20260807 18:05:54.881528 10742 raft_consensus.cc:1261] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Refusing update from remote peer 460eabba634a429e92cda02e78d2d745: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:54.881574 10599 raft_consensus.cc:1261] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Refusing update from remote peer 460eabba634a429e92cda02e78d2d745: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:54.882330 10934 consensus_queue.cc:1035] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:54.882753 10933 consensus_queue.cc:1035] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:54.934721 10285 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:54.934968 10285 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:54.937857 10285 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Sending DeleteTablet for 3 replicas of tablet deab871748734a2fa9585e96390f8646
I20260807 18:05:54.938581 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet deab871748734a2fa9585e96390f8646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34120
I20260807 18:05:54.938753 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet deab871748734a2fa9585e96390f8646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:60512
I20260807 18:05:54.938871 10916 tablet_replica.cc:330] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: stopping tablet replica
I20260807 18:05:54.939033 10578 tablet_service.cc:1514] Processing DeleteTablet for tablet deab871748734a2fa9585e96390f8646 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:54 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:53368
I20260807 18:05:54.939071 10916 raft_consensus.cc:2229] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:54.939306 10918 tablet_replica.cc:330] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: stopping tablet replica
I20260807 18:05:54.939430 10916 raft_consensus.cc:2258] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.939546 10918 raft_consensus.cc:2229] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:54.939781 10917 tablet_replica.cc:330] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: stopping tablet replica
I20260807 18:05:54.939785 10918 raft_consensus.cc:2258] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.940030 10917 raft_consensus.cc:2229] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:54.940246 10917 raft_consensus.cc:2258] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:54.940217 10285 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:05:54.940874 10916 ts_tablet_manager.cc:1905] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:54.940896 10918 ts_tablet_manager.cc:1905] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:05:54.941056 10285 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:05:54.943398 10917 ts_tablet_manager.cc:1905] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:54.947634 10918 ts_tablet_manager.cc:1918] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:54.947803 10918 log.cc:1196] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/deab871748734a2fa9585e96390f8646
I20260807 18:05:54.948100 10425 tablet_service.cc:1467] Processing CreateTablet for tablet 166f8c73889746f3b678d01265004d28 (DEFAULT_TABLE table=test-table [id=74ac3d61a18f4e118911733407ca6ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.948256 10918 ts_tablet_manager.cc:1939] T deab871748734a2fa9585e96390f8646 P fcac58336a2946548ef501b1b0cdce14: Deleting consensus metadata
I20260807 18:05:54.948588 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166f8c73889746f3b678d01265004d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.948586 10578 tablet_service.cc:1467] Processing CreateTablet for tablet 166f8c73889746f3b678d01265004d28 (DEFAULT_TABLE table=test-table [id=74ac3d61a18f4e118911733407ca6ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.948975 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166f8c73889746f3b678d01265004d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.950007 10917 ts_tablet_manager.cc:1918] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:54.950139 10917 log.cc:1196] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/deab871748734a2fa9585e96390f8646
I20260807 18:05:54.950462 10917 ts_tablet_manager.cc:1939] T deab871748734a2fa9585e96390f8646 P 0d72067fa003487cb20e580452a3279e: Deleting consensus metadata
I20260807 18:05:54.951738 10271 catalog_manager.cc:4841] TS fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323): tablet deab871748734a2fa9585e96390f8646 (table test-table [id=ae2570fd97974032a0926a5d6ace6f7a]) successfully deleted
I20260807 18:05:54.952050 10270 catalog_manager.cc:4841] TS 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689): tablet deab871748734a2fa9585e96390f8646 (table test-table [id=ae2570fd97974032a0926a5d6ace6f7a]) successfully deleted
I20260807 18:05:54.952576 10716 tablet_service.cc:1467] Processing CreateTablet for tablet 166f8c73889746f3b678d01265004d28 (DEFAULT_TABLE table=test-table [id=74ac3d61a18f4e118911733407ca6ddf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:05:54.952988 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 166f8c73889746f3b678d01265004d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:05:54.954552 10916 ts_tablet_manager.cc:1918] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:05:54.954780 10916 log.cc:1196] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/deab871748734a2fa9585e96390f8646
I20260807 18:05:54.955341 10916 ts_tablet_manager.cc:1939] T deab871748734a2fa9585e96390f8646 P 460eabba634a429e92cda02e78d2d745: Deleting consensus metadata
I20260807 18:05:54.955585 10927 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Bootstrap starting.
I20260807 18:05:54.955649 10928 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Bootstrap starting.
I20260807 18:05:54.957389 10927 tablet_bootstrap.cc:654] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.957448 10270 catalog_manager.cc:4841] TS 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161): tablet deab871748734a2fa9585e96390f8646 (table test-table [id=ae2570fd97974032a0926a5d6ace6f7a]) successfully deleted
I20260807 18:05:54.957643 10928 tablet_bootstrap.cc:654] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.960130 10929 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Bootstrap starting.
I20260807 18:05:54.960156 10927 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: No bootstrap required, opened a new log
I20260807 18:05:54.960367 10927 ts_tablet_manager.cc:1397] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260807 18:05:54.961158 10928 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: No bootstrap required, opened a new log
I20260807 18:05:54.961238 10927 raft_consensus.cc:348] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.961396 10928 ts_tablet_manager.cc:1397] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260807 18:05:54.961495 10927 raft_consensus.cc:374] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.961589 10927 raft_consensus.cc:729] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.961862 10929 tablet_bootstrap.cc:654] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:54.961902 10927 consensus_queue.cc:260] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.962298 10928 raft_consensus.cc:348] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.962540 10928 raft_consensus.cc:374] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.962657 10928 raft_consensus.cc:729] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460eabba634a429e92cda02e78d2d745, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.962909 10928 consensus_queue.cc:260] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.963282 10927 ts_tablet_manager.cc:1428] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 18:05:54.963443 10928 ts_tablet_manager.cc:1428] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260807 18:05:54.965871 10929 tablet_bootstrap.cc:492] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: No bootstrap required, opened a new log
I20260807 18:05:54.966056 10929 ts_tablet_manager.cc:1397] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:05:54.966892 10929 raft_consensus.cc:348] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.967077 10929 raft_consensus.cc:374] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:54.967182 10929 raft_consensus.cc:729] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d72067fa003487cb20e580452a3279e, State: Initialized, Role: FOLLOWER
I20260807 18:05:54.967446 10929 consensus_queue.cc:260] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:54.968676 10929 ts_tablet_manager.cc:1428] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 18:05:55.174888 10909 raft_consensus.cc:482] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:05:55.175158 10909 raft_consensus.cc:504] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:55.175958 10909 leader_election.cc:288] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:55.176770 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166f8c73889746f3b678d01265004d28" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745" is_pre_election: true
I20260807 18:05:55.176848 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166f8c73889746f3b678d01265004d28" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e" is_pre_election: true
I20260807 18:05:55.177054 10448 raft_consensus.cc:2454] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 0.
I20260807 18:05:55.177210 10742 raft_consensus.cc:2454] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 0.
I20260807 18:05:55.177629 10524 leader_election.cc:302] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460eabba634a429e92cda02e78d2d745, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:55.177923 10909 raft_consensus.cc:2781] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:05:55.178052 10909 raft_consensus.cc:482] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:05:55.178151 10909 raft_consensus.cc:3037] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:55.180212 10909 raft_consensus.cc:504] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:55.180914 10909 leader_election.cc:288] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 election: Requested vote from peers 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161), 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689)
I20260807 18:05:55.181478 10742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166f8c73889746f3b678d01265004d28" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d72067fa003487cb20e580452a3279e"
I20260807 18:05:55.181480 10448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "166f8c73889746f3b678d01265004d28" candidate_uuid: "fcac58336a2946548ef501b1b0cdce14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460eabba634a429e92cda02e78d2d745"
I20260807 18:05:55.181718 10742 raft_consensus.cc:3037] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:55.181825 10448 raft_consensus.cc:3037] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:55.183938 10742 raft_consensus.cc:2454] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 1.
I20260807 18:05:55.184279 10448 raft_consensus.cc:2454] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fcac58336a2946548ef501b1b0cdce14 in term 1.
I20260807 18:05:55.184489 10524 leader_election.cc:302] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d72067fa003487cb20e580452a3279e, fcac58336a2946548ef501b1b0cdce14; no voters:
I20260807 18:05:55.184885 10909 raft_consensus.cc:2781] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:55.185302 10909 raft_consensus.cc:686] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 1 LEADER]: Becoming Leader. State: Replica: fcac58336a2946548ef501b1b0cdce14, State: Running, Role: LEADER
I20260807 18:05:55.185595 10909 consensus_queue.cc:237] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } }
I20260807 18:05:55.188339 10284 catalog_manager.cc:5495] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 reported cstate change: term changed from 0 to 1, leader changed from <none> to fcac58336a2946548ef501b1b0cdce14 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "fcac58336a2946548ef501b1b0cdce14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcac58336a2946548ef501b1b0cdce14" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260807 18:05:55.221210 10949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:55.215879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.8.62.130:46323)
0807 18:05:55.216437 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:55.216441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:55.216463 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:05:55.216523 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:55.216526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:55.216659 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:55.216753 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:05:55.217105 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:55.217912 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:55.219270 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:55.219735 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:55.219911 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:55.220308 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:05:55.220418 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:05:55.220488 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:55.220772 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:05:55.220836 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":337,"threads_started":1}
I20260807 18:05:55.231389 10742 raft_consensus.cc:1261] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Refusing update from remote peer fcac58336a2946548ef501b1b0cdce14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:55.231459 10448 raft_consensus.cc:1261] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Refusing update from remote peer fcac58336a2946548ef501b1b0cdce14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:05:55.232038 10947 consensus_queue.cc:1035] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d72067fa003487cb20e580452a3279e" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:05:55.232438 10909 consensus_queue.cc:1035] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460eabba634a429e92cda02e78d2d745" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 33161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 18:05:55.252861 10897 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 18:05:56.865049 10283 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:56.865303 10283 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:05:56.873884 10283 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ce1e50c05fc485eb5da6f550c49fffc: Sending DeleteTablet for 3 replicas of tablet 166f8c73889746f3b678d01265004d28
I20260807 18:05:56.875092 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet 166f8c73889746f3b678d01265004d28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:56 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34120
I20260807 18:05:56.875142 10578 tablet_service.cc:1514] Processing DeleteTablet for tablet 166f8c73889746f3b678d01265004d28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:56 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:53368
I20260807 18:05:56.875639 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet 166f8c73889746f3b678d01265004d28 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:05:56 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:60512
I20260807 18:05:56.876192 10974 tablet_replica.cc:330] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: stopping tablet replica
I20260807 18:05:56.876513 10974 raft_consensus.cc:2229] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:56.876858 10974 raft_consensus.cc:2258] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:56.878790 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10358
I20260807 18:05:56.882966 10974 ts_tablet_manager.cc:1905] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:56.893714 10976 tablet_replica.cc:330] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: stopping tablet replica
I20260807 18:05:56.893973 10976 raft_consensus.cc:2229] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:05:56.894260 10976 raft_consensus.cc:2258] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:56.895249 10976 ts_tablet_manager.cc:1905] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:56.900619 10974 ts_tablet_manager.cc:1918] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:56.900840 10974 log.cc:1196] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/166f8c73889746f3b678d01265004d28
I20260807 18:05:56.901520 10974 ts_tablet_manager.cc:1939] T 166f8c73889746f3b678d01265004d28 P 460eabba634a429e92cda02e78d2d745: Deleting consensus metadata
I20260807 18:05:56.903079 10270 catalog_manager.cc:4841] TS 460eabba634a429e92cda02e78d2d745 (127.8.62.129:33161): tablet 166f8c73889746f3b678d01265004d28 (table test-table [id=74ac3d61a18f4e118911733407ca6ddf]) successfully deleted
I20260807 18:05:56.903620 10975 tablet_replica.cc:330] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: stopping tablet replica
I20260807 18:05:56.903954 10975 raft_consensus.cc:2229] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:05:56.904093 10976 ts_tablet_manager.cc:1918] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:56.904322 10976 log.cc:1196] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/166f8c73889746f3b678d01265004d28
I20260807 18:05:56.904595 10975 raft_consensus.cc:2258] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:05:56.904958 10976 ts_tablet_manager.cc:1939] T 166f8c73889746f3b678d01265004d28 P 0d72067fa003487cb20e580452a3279e: Deleting consensus metadata
I20260807 18:05:56.905943 10975 ts_tablet_manager.cc:1905] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:05:56.906383 10270 catalog_manager.cc:4841] TS 0d72067fa003487cb20e580452a3279e (127.8.62.131:38689): tablet 166f8c73889746f3b678d01265004d28 (table test-table [id=74ac3d61a18f4e118911733407ca6ddf]) successfully deleted
Aug 07 18:05:56 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:05:56.912765 10975 ts_tablet_manager.cc:1918] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 18:05:56.912948 10975 log.cc:1196] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/166f8c73889746f3b678d01265004d28
I20260807 18:05:56.913476 10975 ts_tablet_manager.cc:1939] T 166f8c73889746f3b678d01265004d28 P fcac58336a2946548ef501b1b0cdce14: Deleting consensus metadata
I20260807 18:05:56.914732 10271 catalog_manager.cc:4841] TS fcac58336a2946548ef501b1b0cdce14 (127.8.62.130:46323): tablet 166f8c73889746f3b678d01265004d28 (table test-table [id=74ac3d61a18f4e118911733407ca6ddf]) successfully deleted
I20260807 18:05:56.917266 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:56.880431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.8.62.129:33161)
0807 18:05:56.893253 (+ 12822us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:56.893261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:56.893289 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:05:56.893360 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:56.893365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:56.893503 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:56.893710 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:56.896694 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.898038 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.900988 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.901886 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.902041 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:56.908985 (+ 6944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:56.909002 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:56.909018 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:56.909059 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:56.913844 (+ 4785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:56.914942 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:56.914954 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:56.914957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:56.915120 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:56.915551 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:56.915557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:56.915560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:56.915946 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:56.916259 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:56.916641 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:05:56.916903 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12598,"thread_start_us":12902,"threads_started":1}
W20260807 18:05:56.918021 10495 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34150
W20260807 18:05:56.918682 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10358: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:56.918911 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10358
I20260807 18:05:56.944764 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10505
Aug 07 18:05:56 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:05:56.961387 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:56.945603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.8.62.130:46323)
0807 18:05:56.946192 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:56.946197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:56.946213 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:05:56.946268 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:56.946271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:56.946329 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:56.946424 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:56.947666 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.948434 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.949815 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.950259 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.950369 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:56.954710 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:56.954726 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:56.954743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:56.954781 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:56.958141 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:56.959006 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:56.959018 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:56.959020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:56.959187 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:56.959671 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:56.959676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:56.959679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:56.960075 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:56.960391 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:56.960909 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 18:05:56.961106 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":306,"threads_started":1}
W20260807 18:05:56.962182 10642 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53388
W20260807 18:05:56.962874 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10505: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:56.963037 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10505
I20260807 18:05:56.987874 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10651
Aug 07 18:05:56 dist-test-slave-rnrw krb5kdc[10238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:05:57.005565 10980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:56.988762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.8.62.131:38689)
0807 18:05:56.989315 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:56.989319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:56.989337 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:05:56.989393 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:56.989396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:56.989462 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:56.989565 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:56.991093 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.991888 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.993216 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:56.993617 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:56.993714 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:56.998801 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:56.998825 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:56.998842 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:56.998911 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:57.002334 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:57.003177 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:57.003184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:57.003187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:57.003404 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:57.003863 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:57.003867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:57.003870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:57.004282 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:57.004704 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:57.005051 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 18:05:57.005264 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":304,"threads_started":1}
W20260807 18:05:57.006462 10788 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60534
W20260807 18:05:57.007205 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10651: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:05:57.007359 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10651
I20260807 18:05:57.030933 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10251
I20260807 18:05:57.032176 10326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:05:57.152750 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10251
2026-08-07T18:05:57Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9310 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:05:57 dist-test-slave-rnrw krb5kdc[10985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:05:57 dist-test-slave-rnrw krb5kdc[10985](info): set up 2 sockets
Aug 07 18:05:57 dist-test-slave-rnrw krb5kdc[10985](info): commencing operation
krb5kdc: starting...
W20260807 18:05:58.275406 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.007s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:05:58 dist-test-slave-rnrw krb5kdc[10985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:05:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:05:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:05:58.484680 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44733
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:58.822314 10998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:58.822603 10998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:58.822788 10998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:58.830251 10998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:05:58.830370 10998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:58.830475 10998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:58.830574 10998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:05:58.830662 10998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:05:58.840762 10998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:58.842134 10998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:58.843637 10998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:58.861177 11003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:58.861151 11006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:58.861097 11004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:58.861446 10998 server_base.cc:1034] running on GCE node
I20260807 18:05:58.862543 10998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:58.864050 10998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:58.865274 10998 hybrid_clock.cc:648] HybridClock initialized: now 1786125958865217 us; error 45 us; skew 500 ppm
I20260807 18:05:58.865684 10998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:58 dist-test-slave-rnrw krb5kdc[10985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125958, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:58.869192 10998 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:05:58.870903 10998 webserver.cc:460] Webserver started at http://127.8.62.190:33285/ using document root <none> and password file <none>
I20260807 18:05:58.871701 10998 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:58.871832 10998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:58.872192 10998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:58.875425 10998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "aac8aa8622424c40bd811a3934a1e3d1"
format_stamp: "Formatted at 2026-08-07 18:05:58 on dist-test-slave-rnrw"
server_key: "09f03785ad20375b78495359843eeff0"
server_key_iv: "916f4e2cde0a4f8c3dd5ce529384e9b5"
server_key_version: "encryptionkey@0"
I20260807 18:05:58.876291 10998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "aac8aa8622424c40bd811a3934a1e3d1"
format_stamp: "Formatted at 2026-08-07 18:05:58 on dist-test-slave-rnrw"
server_key: "09f03785ad20375b78495359843eeff0"
server_key_iv: "916f4e2cde0a4f8c3dd5ce529384e9b5"
server_key_version: "encryptionkey@0"
I20260807 18:05:58.881979 10998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:05:58.885761 11014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:58.887251 10998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:05:58.887449 10998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "aac8aa8622424c40bd811a3934a1e3d1"
format_stamp: "Formatted at 2026-08-07 18:05:58 on dist-test-slave-rnrw"
server_key: "09f03785ad20375b78495359843eeff0"
server_key_iv: "916f4e2cde0a4f8c3dd5ce529384e9b5"
server_key_version: "encryptionkey@0"
I20260807 18:05:58.887673 10998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:58.927708 10998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:58.932188 10998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:58.932512 10998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:58.952387 10998 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:44733
I20260807 18:05:58.952423 11068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:44733 every 8 connection(s)
I20260807 18:05:58.953897 10998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:05:58.955148 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 10998
I20260807 18:05:58.955386 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:05:58.956022 8442 external_mini_cluster.cc:1428] Setting key 23da1daf870a1d7152637973ae14c5da
I20260807 18:05:58.958279 11069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:05:58 dist-test-slave-rnrw krb5kdc[10985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:05:58.972878 11069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1: Bootstrap starting.
I20260807 18:05:58.976078 11069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1: Neither blocks nor log segments found. Creating new log.
I20260807 18:05:58.977221 11069 log.cc:824] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:05:58.980502 11069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1: No bootstrap required, opened a new log
I20260807 18:05:58.980679 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:58.958368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34888 (local address 127.8.62.190:44733)
0807 18:05:58.959332 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:58.959345 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:58.959387 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:05:58.961164 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:58.961187 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:58.961583 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:58.961935 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:58.963818 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:58.965282 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:58.967092 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:58.967814 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:58.967957 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:58.972895 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:58.972904 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:58.972920 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:58.972977 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:58.976102 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:58.976943 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:58.976949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:58.976956 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:58.977126 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:58.977560 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:58.977565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:58.977570 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:58.978430 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:58.978724 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:58.979183 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:05:58.979567 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":435,"threads_started":1}
I20260807 18:05:58.988296 11069 raft_consensus.cc:348] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } }
I20260807 18:05:58.988777 11069 raft_consensus.cc:374] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:05:58.988893 11069 raft_consensus.cc:729] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aac8aa8622424c40bd811a3934a1e3d1, State: Initialized, Role: FOLLOWER
I20260807 18:05:58.989564 11069 consensus_queue.cc:260] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } }
I20260807 18:05:58.989838 11069 raft_consensus.cc:388] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:05:58.990012 11069 raft_consensus.cc:482] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:05:58.990190 11069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:05:58.992774 11069 raft_consensus.cc:504] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } }
I20260807 18:05:58.993461 11069 leader_election.cc:302] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aac8aa8622424c40bd811a3934a1e3d1; no voters:
I20260807 18:05:58.994168 11069 leader_election.cc:288] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:05:58.994325 11074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:05:58.995038 11074 raft_consensus.cc:686] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [term 1 LEADER]: Becoming Leader. State: Replica: aac8aa8622424c40bd811a3934a1e3d1, State: Running, Role: LEADER
I20260807 18:05:58.995643 11074 consensus_queue.cc:237] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } }
I20260807 18:05:58.996460 11069 sys_catalog.cc:564] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:05:58.998746 11076 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aac8aa8622424c40bd811a3934a1e3d1. Latest consensus state: current_term: 1 leader_uuid: "aac8aa8622424c40bd811a3934a1e3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } } }
I20260807 18:05:58.999310 11076 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:59.000257 11082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:05:59.000173 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aac8aa8622424c40bd811a3934a1e3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aac8aa8622424c40bd811a3934a1e3d1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44733 } } }
I20260807 18:05:59.005036 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:05:59.008427 11082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:05:59.020797 11082 catalog_manager.cc:1347] Generated new cluster ID: 1543b95f90794d24a463a8e970dfe63b
I20260807 18:05:59.020927 11082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:05:59.043642 11082 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:05:59.045117 11082 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:05:59.063308 11082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aac8aa8622424c40bd811a3934a1e3d1: Generated new TSK 0
I20260807 18:05:59.064257 11082 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:05:59.116930 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:59.467198 11096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:59.467571 11096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:59.467751 11096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:59.475291 11096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:59.475451 11096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:59.475643 11096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:05:59.485571 11096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:59.487224 11096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:59.488734 11096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:05:59.504374 11103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:59.504401 11102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:05:59.504496 11105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:05:59.506367 11096 server_base.cc:1034] running on GCE node
I20260807 18:05:59.507215 11096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:05:59.508445 11096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:05:59.509765 11096 hybrid_clock.cc:648] HybridClock initialized: now 1786125959509707 us; error 47 us; skew 500 ppm
I20260807 18:05:59.510138 11096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:05:59 dist-test-slave-rnrw krb5kdc[10985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125959, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:05:59.513654 11096 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:05:59.515493 11096 webserver.cc:460] Webserver started at http://127.8.62.129:39227/ using document root <none> and password file <none>
I20260807 18:05:59.516299 11096 fs_manager.cc:359] Metadata directory not provided
I20260807 18:05:59.516471 11096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:05:59.516852 11096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:05:59.520092 11096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "2465824270234ee9952b70edc4dd59b5"
format_stamp: "Formatted at 2026-08-07 18:05:59 on dist-test-slave-rnrw"
server_key: "ce8c0f81e7e2565555d776668fc7d3a4"
server_key_iv: "7ab227bbb80e0f3fe8564c09b7d37353"
server_key_version: "encryptionkey@0"
I20260807 18:05:59.521031 11096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "2465824270234ee9952b70edc4dd59b5"
format_stamp: "Formatted at 2026-08-07 18:05:59 on dist-test-slave-rnrw"
server_key: "ce8c0f81e7e2565555d776668fc7d3a4"
server_key_iv: "7ab227bbb80e0f3fe8564c09b7d37353"
server_key_version: "encryptionkey@0"
I20260807 18:05:59.526957 11096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:05:59.530848 11112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:59.532336 11096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:05:59.532662 11096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "2465824270234ee9952b70edc4dd59b5"
format_stamp: "Formatted at 2026-08-07 18:05:59 on dist-test-slave-rnrw"
server_key: "ce8c0f81e7e2565555d776668fc7d3a4"
server_key_iv: "7ab227bbb80e0f3fe8564c09b7d37353"
server_key_version: "encryptionkey@0"
I20260807 18:05:59.532908 11096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:05:59.545063 11096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:05:59.549626 11096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:05:59.549932 11096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:05:59.550932 11096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:05:59.552299 11096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:05:59.552403 11096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:05:59.552589 11096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:05:59.552680 11096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:05:59.586552 11096 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:43363
I20260807 18:05:59.586611 11234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:43363 every 8 connection(s)
I20260807 18:05:59.588016 11096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:05:59.590082 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11096
I20260807 18:05:59.590315 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:05:59.591141 8442 external_mini_cluster.cc:1428] Setting key e4a625abcdc87c7f7ffd5c4ca5edf98e
Aug 07 18:05:59 dist-test-slave-rnrw krb5kdc[10985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125959, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:05:59.610895 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:59.592906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:45139 (local address 127.8.62.190:44733)
0807 18:05:59.593895 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0807 18:05:59.593902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:05:59.593925 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:05:59.595325 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:05:59.595331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:05:59.595415 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:05:59.595555 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:05:59.597236 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:59.598001 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:59.600102 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:05:59.600502 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:05:59.600681 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:59.604836 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:05:59.604841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:05:59.604844 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:05:59.604875 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:05:59.607685 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:59.608437 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:59.608448 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:59.608450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:59.608632 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:05:59.609072 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:05:59.609077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:05:59.609080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:05:59.609370 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:05:59.609532 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:05:59.610286 (+ 754us) server_negotiation.cc:299] Negotiation successful
0807 18:05:59.610525 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":748,"threads_started":1}
I20260807 18:05:59.612170 11237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:05:59.592094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44733 (local address 127.8.62.129:45139)
0807 18:05:59.593178 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0807 18:05:59.593230 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:05:59.595051 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:05:59.595793 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:05:59.595802 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:05:59.596325 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:05:59.597060 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:59.597092 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:59.598173 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:59.598183 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:59.599923 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:05:59.599944 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:05:59.600775 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:05:59.600785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:05:59.601128 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:05:59.601865 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:05:59.601899 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:05:59.604609 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:05:59.607840 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:59.607850 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:59.607867 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:59.608301 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:59.608745 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:05:59.608750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:05:59.608753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:05:59.608948 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:05:59.609504 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:05:59.609512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:05:59.609894 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 18:05:59.610168 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 18:05:59.610723 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":490,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:05:59.614078 11235 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44733
I20260807 18:05:59.614420 11235 heartbeater.cc:464] Registering TS with master...
I20260807 18:05:59.615180 11235 heartbeater.cc:511] Master 127.8.62.190:44733 requested a full tablet report, sending...
I20260807 18:05:59.617651 11031 ts_manager.cc:194] Registered new tserver with Master: 2465824270234ee9952b70edc4dd59b5 (127.8.62.129:43363)
I20260807 18:05:59.619374 11031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:45139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:05:59.640528 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:05:59.972334 11241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:05:59.972748 11241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:05:59.972940 11241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:05:59.980326 11241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:05:59.980441 11241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:05:59.980675 11241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:05:59.990820 11241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:05:59.992159 11241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:05:59.993521 11241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:00.008219 11246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:00.008179 11247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:00.008276 11249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:00.009100 11241 server_base.cc:1034] running on GCE node
I20260807 18:06:00.009793 11241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:00.010802 11241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:00.012006 11241 hybrid_clock.cc:648] HybridClock initialized: now 1786125960011959 us; error 36 us; skew 500 ppm
I20260807 18:06:00.012344 11241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[10985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125960, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:00.020776 11241 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:00.022439 11241 webserver.cc:460] Webserver started at http://127.8.62.130:43965/ using document root <none> and password file <none>
I20260807 18:06:00.023187 11241 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:00.023322 11241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:00.023658 11241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:00.026671 11241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "ce7a7eb0ea00434489e4dfb69de3254d"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "023939ae356cf5e11b8192a0d82233d6"
server_key_iv: "04ee6c2b192c6db3bb1e3843cb6ed5c4"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.027498 11241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "ce7a7eb0ea00434489e4dfb69de3254d"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "023939ae356cf5e11b8192a0d82233d6"
server_key_iv: "04ee6c2b192c6db3bb1e3843cb6ed5c4"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.032918 11241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:06:00.036669 11256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.038223 11241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:00.038406 11241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "ce7a7eb0ea00434489e4dfb69de3254d"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "023939ae356cf5e11b8192a0d82233d6"
server_key_iv: "04ee6c2b192c6db3bb1e3843cb6ed5c4"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.038614 11241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:00.061277 11241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:00.066177 11241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:00.066510 11241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:00.067677 11241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:00.069206 11241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:00.069283 11241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.069428 11241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:00.069481 11241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.105159 11241 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:39473
I20260807 18:06:00.105213 11373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:39473 every 8 connection(s)
I20260807 18:06:00.106534 11241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:00.110706 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11241
I20260807 18:06:00.110924 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:00.111438 8442 external_mini_cluster.cc:1428] Setting key 281313841f46dfcb31abb88af20819fc
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[10985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125960, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:00.129351 11238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:00.109410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:48469 (local address 127.8.62.190:44733)
0807 18:06:00.109618 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:00.109623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:00.111061 (+ 1438us) server_negotiation.cc:407] Connection header received
0807 18:06:00.113496 (+ 2435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:00.113502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:00.113606 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:00.113727 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:00.115470 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.116233 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.118067 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.118555 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.118708 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:00.123256 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:00.123261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:00.123264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:00.123298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:00.126066 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:00.126888 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:00.126892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:00.126895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:00.127025 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:00.127501 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:00.127510 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:00.127512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:00.127788 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:00.127964 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:00.128740 (+ 776us) server_negotiation.cc:299] Negotiation successful
0807 18:06:00.128991 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:06:00.130577 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:00.109921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44733 (local address 127.8.62.130:48469)
0807 18:06:00.110755 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:00.110849 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:00.113139 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:00.114016 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:00.114025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:00.114621 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:00.115258 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:00.115282 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.116412 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.116418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:00.117876 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:00.117889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.118826 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.118834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:00.119109 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:00.119893 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:00.119943 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:00.122995 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:00.126233 (+ 3238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:00.126244 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:00.126264 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:00.126738 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:00.127154 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:00.127159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:00.127163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:00.127358 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:00.127941 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:00.127949 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:00.128321 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 18:06:00.128622 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 18:06:00.129207 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":341,"threads_started":1}
I20260807 18:06:00.132341 11374 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44733
I20260807 18:06:00.132817 11374 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:06:00.133684 11374 heartbeater.cc:511] Master 127.8.62.190:44733 requested a full tablet report, sending...
I20260807 18:06:00.135387 11031 ts_manager.cc:194] Registered new tserver with Master: ce7a7eb0ea00434489e4dfb69de3254d (127.8.62.130:39473)
I20260807 18:06:00.136309 11031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:48469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:00.159766 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:00.504199 11379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:00.504495 11379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:00.504654 11379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:00.512282 11379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:00.512403 11379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:00.512534 11379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:00.522821 11379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:00.524075 11379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:00.525403 11379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:00.539348 11385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:00.539402 11384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:00.540014 11387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:00.540238 11379 server_base.cc:1034] running on GCE node
I20260807 18:06:00.540957 11379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:00.541985 11379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:00.543226 11379 hybrid_clock.cc:648] HybridClock initialized: now 1786125960543165 us; error 71 us; skew 500 ppm
I20260807 18:06:00.543613 11379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[10985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125960, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:00.546957 11379 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:00.548612 11379 webserver.cc:460] Webserver started at http://127.8.62.131:46453/ using document root <none> and password file <none>
I20260807 18:06:00.549331 11379 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:00.549435 11379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:00.549697 11379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:00.552655 11379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "9b75a7dc95e14183bdadeed83884e388"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "8b64a6f99b5290e883bf8a1e43ce407b"
server_key_iv: "99cde88d5f7222f231c9e5d1fe05ca2f"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.553390 11379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "9b75a7dc95e14183bdadeed83884e388"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "8b64a6f99b5290e883bf8a1e43ce407b"
server_key_iv: "99cde88d5f7222f231c9e5d1fe05ca2f"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.558430 11379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:06:00.561980 11394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.563436 11379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:00.563575 11379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "9b75a7dc95e14183bdadeed83884e388"
format_stamp: "Formatted at 2026-08-07 18:06:00 on dist-test-slave-rnrw"
server_key: "8b64a6f99b5290e883bf8a1e43ce407b"
server_key_iv: "99cde88d5f7222f231c9e5d1fe05ca2f"
server_key_version: "encryptionkey@0"
I20260807 18:06:00.563752 11379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:00.589499 11379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:00.594952 11379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:00.595316 11379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:00.596484 11379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:00.597980 11379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:00.598093 11379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.598217 11379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:00.598301 11379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:00.623378 11235 heartbeater.cc:503] Master 127.8.62.190:44733 was elected leader, sending a full tablet report...
I20260807 18:06:00.631551 11379 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:38625
I20260807 18:06:00.631588 11510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:38625 every 8 connection(s)
I20260807 18:06:00.632998 11379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:00.640681 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11379
I20260807 18:06:00.640905 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:00.641328 8442 external_mini_cluster.cc:1428] Setting key a14e8cd3b178bac2a995a03469e46a51
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[10985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125960, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:00.654279 11513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:00.635696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:37629 (local address 127.8.62.190:44733)
0807 18:06:00.636167 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:00.636171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:00.637339 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 18:06:00.639267 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:00.639272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:00.639361 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:00.639493 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:00.641227 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.642207 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.644308 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.644808 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.644949 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:00.648642 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:00.648648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:00.648653 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:00.648694 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:00.651376 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:00.652110 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:00.652114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:00.652116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:00.652244 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:00.652737 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:00.652740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:00.652742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:00.653007 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:00.653121 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:00.653810 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 18:06:00.653985 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":230,"threads_started":1}
I20260807 18:06:00.655524 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:00.636298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44733 (local address 127.8.62.131:37629)
0807 18:06:00.637168 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:00.637211 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:00.638975 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:00.639742 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:00.639751 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:00.640296 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:00.641047 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:00.641066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.642387 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.642399 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:00.644147 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:00.644165 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:00.645044 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:00.645053 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:00.645297 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:00.646033 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:00.646058 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:00.648365 (+ 2307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:00.651520 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:00.651528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:00.651542 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:00.651998 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:00.652340 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:00.652343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:00.652345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:00.652519 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:00.653114 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:00.653120 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:00.653454 (+ 334us) client_negotiation.cc:769] Sending connection context
0807 18:06:00.653684 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 18:06:00.654255 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":365,"threads_started":1}
I20260807 18:06:00.657115 11511 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44733
I20260807 18:06:00.657405 11511 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:00.658069 11511 heartbeater.cc:511] Master 127.8.62.190:44733 requested a full tablet report, sending...
I20260807 18:06:00.659405 11031 ts_manager.cc:194] Registered new tserver with Master: 9b75a7dc95e14183bdadeed83884e388 (127.8.62.131:38625)
I20260807 18:06:00.660257 11031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:37629
I20260807 18:06:00.669638 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:06:00.703543 11513 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:00.700790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.8.62.190:44733)
0807 18:06:00.701078 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:00.701084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:00.701448 (+ 364us) server_negotiation.cc:407] Connection header received
0807 18:06:00.701706 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:00.701715 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:00.701803 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:00.701916 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:00.703334 (+ 1418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34890: BlockingRecv error: recv got EOF from 127.0.0.1:34890 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 18:06:00.706779 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11096
W20260807 18:06:00.709707 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11096: Not authorized: Client connection negotiation failed: client connection to 127.8.62.129:43363: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:06:00.709899 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11096
I20260807 18:06:00.727074 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11241
W20260807 18:06:00.729795 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11241: Not authorized: Client connection negotiation failed: client connection to 127.8.62.130:39473: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:06:00.729988 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11241
I20260807 18:06:00.747061 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11379
W20260807 18:06:00.750209 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11379: Not authorized: Client connection negotiation failed: client connection to 127.8.62.131:38625: server requires authentication, but client does not have Kerberos credentials available
I20260807 18:06:00.750334 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11379
I20260807 18:06:00.766952 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 10998
I20260807 18:06:00.768249 11063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:00.877254 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 10998
2026-08-07T18:06:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3721 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[11529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[11529](info): set up 2 sockets
Aug 07 18:06:00 dist-test-slave-rnrw krb5kdc[11529](info): commencing operation
krb5kdc: starting...
W20260807 18:06:01.999492 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.078s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:02.208895 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44671
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:34903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:02.543735 11542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:02.544052 11542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:02.544227 11542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:02.551826 11542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:02.551936 11542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:02.552029 11542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:02.552137 11542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:02.552230 11542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:02.562505 11542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:34903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:02.563828 11542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:02.565183 11542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:02.577322 11548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:02.577417 11547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:02.577440 11550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:02.578573 11542 server_base.cc:1034] running on GCE node
I20260807 18:06:02.579437 11542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:02.581051 11542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:02.582298 11542 hybrid_clock.cc:648] HybridClock initialized: now 1786125962582242 us; error 43 us; skew 500 ppm
I20260807 18:06:02.582665 11542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125962, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:02.585958 11542 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:02.587445 11542 webserver.cc:460] Webserver started at http://127.8.62.190:42161/ using document root <none> and password file <none>
I20260807 18:06:02.588117 11542 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:02.588227 11542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:02.588493 11542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:02.591615 11542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "a4b94ec0dc094dc0a60a2ab426d434b7"
format_stamp: "Formatted at 2026-08-07 18:06:02 on dist-test-slave-rnrw"
server_key: "4cf44f20702180648e325557bc45a65b"
server_key_iv: "dec9c74e0b8607bf765d59b8e4fc872a"
server_key_version: "encryptionkey@0"
I20260807 18:06:02.592399 11542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "a4b94ec0dc094dc0a60a2ab426d434b7"
format_stamp: "Formatted at 2026-08-07 18:06:02 on dist-test-slave-rnrw"
server_key: "4cf44f20702180648e325557bc45a65b"
server_key_iv: "dec9c74e0b8607bf765d59b8e4fc872a"
server_key_version: "encryptionkey@0"
I20260807 18:06:02.597994 11542 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:06:02.602087 11558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:02.603991 11542 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:06:02.604189 11542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "a4b94ec0dc094dc0a60a2ab426d434b7"
format_stamp: "Formatted at 2026-08-07 18:06:02 on dist-test-slave-rnrw"
server_key: "4cf44f20702180648e325557bc45a65b"
server_key_iv: "dec9c74e0b8607bf765d59b8e4fc872a"
server_key_version: "encryptionkey@0"
I20260807 18:06:02.604416 11542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:02.628970 11542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:02.633291 11542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:02.633597 11542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:02.652771 11542 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:44671
I20260807 18:06:02.652806 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:44671 every 8 connection(s)
I20260807 18:06:02.654299 11542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:02.658136 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:02.658984 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11542
I20260807 18:06:02.659198 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:02.659865 8442 external_mini_cluster.cc:1428] Setting key 66de650a5a0baa4ea4187f7d966f8c71
I20260807 18:06:02.673134 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7: Bootstrap starting.
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:02.676537 11610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:02.678015 11610 log.cc:824] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:02.681632 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7: No bootstrap required, opened a new log
I20260807 18:06:02.685506 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:02.661907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42592 (local address 127.8.62.190:44671)
0807 18:06:02.662850 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:02.662867 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:02.662920 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:06:02.664942 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:02.664973 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:02.665555 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:02.666040 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:02.667448 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:02.669517 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:02.670869 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:02.671974 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:02.672182 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:02.676202 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:02.676214 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:02.676236 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:02.676293 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:02.680218 (+ 3925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:02.681091 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:02.681101 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:02.681110 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:02.681353 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:02.681759 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:02.681764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:02.681767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:02.682591 (+ 824us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:02.682902 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:02.683388 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:06:02.683908 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":402,"threads_started":1}
I20260807 18:06:02.690104 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:06:02.691778 11610 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } }
I20260807 18:06:02.692173 11610 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:02.692296 11610 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4b94ec0dc094dc0a60a2ab426d434b7, State: Initialized, Role: FOLLOWER
I20260807 18:06:02.692986 11610 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } }
I20260807 18:06:02.693219 11610 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:02.693353 11610 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:02.693534 11610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:02.696000 11610 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } }
I20260807 18:06:02.696611 11610 leader_election.cc:302] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4b94ec0dc094dc0a60a2ab426d434b7; no voters:
I20260807 18:06:02.697335 11610 leader_election.cc:288] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:02.697446 11616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:02.698668 11616 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [term 1 LEADER]: Becoming Leader. State: Replica: a4b94ec0dc094dc0a60a2ab426d434b7, State: Running, Role: LEADER
I20260807 18:06:02.699328 11616 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } }
I20260807 18:06:02.699807 11610 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:02.702390 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4b94ec0dc094dc0a60a2ab426d434b7. Latest consensus state: current_term: 1 leader_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } } }
I20260807 18:06:02.702817 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:02.703490 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4b94ec0dc094dc0a60a2ab426d434b7" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44671 } } }
I20260807 18:06:02.703838 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:02.714248 11621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:02.720587 11621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:02.730546 11621 catalog_manager.cc:1347] Generated new cluster ID: a07397a5565843599764a60b58f2904f
I20260807 18:06:02.730756 11621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:02.745843 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:02.734026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42602 (local address 127.8.62.190:44671)
0807 18:06:02.734221 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:02.734227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:02.737242 (+ 3015us) server_negotiation.cc:407] Connection header received
0807 18:06:02.737445 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:02.737451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:02.737538 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:02.737644 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:02.738959 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:02.740017 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:02.741931 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:02.742582 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:02.742695 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:02.743950 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:02.743956 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:02.743961 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:02.745005 (+ 1044us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:02.745083 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:02.745118 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:02.745232 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:02.745402 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:06:02.745503 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:06:02.750669 11621 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:02.751981 11621 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:02.759907 11621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4b94ec0dc094dc0a60a2ab426d434b7: Generated new TSK 0
I20260807 18:06:02.761013 11621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:06:02.771167 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11542
I20260807 18:06:02.772338 11605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:02.866374 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11542
2026-08-07T18:06:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1982 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11645](info): set up 2 sockets
Aug 07 18:06:02 dist-test-slave-rnrw krb5kdc[11645](info): commencing operation
krb5kdc: starting...
W20260807 18:06:03.971406 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:04 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:04.189275 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36625
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:39643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:36625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:04.534801 11658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:04.535104 11658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:04.535264 11658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:04.542514 11658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:04.542891 11658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:04.543035 11658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:04.543133 11658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:04.543196 11658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:04.553231 11658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:36625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:04.554644 11658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:04.556061 11658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:04.568463 11663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:04.568513 11666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:04.568430 11664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:04.569634 11658 server_base.cc:1034] running on GCE node
I20260807 18:06:04.570562 11658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:04.572156 11658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:04.573421 11658 hybrid_clock.cc:648] HybridClock initialized: now 1786125964573369 us; error 60 us; skew 500 ppm
I20260807 18:06:04.573840 11658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:04 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125964, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:04.577478 11658 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:04.579111 11658 webserver.cc:460] Webserver started at http://127.8.62.190:44247/ using document root <none> and password file <none>
I20260807 18:06:04.579932 11658 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:04.580098 11658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:04.580404 11658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:04.583734 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "a3341c3c6bcb4999bd311f33f8033f77"
format_stamp: "Formatted at 2026-08-07 18:06:04 on dist-test-slave-rnrw"
server_key: "e83d186b33d318f0fdb306e5fcaf3b6e"
server_key_iv: "2abed5a4e540ed2a0b5619552794a077"
server_key_version: "encryptionkey@0"
I20260807 18:06:04.584504 11658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "a3341c3c6bcb4999bd311f33f8033f77"
format_stamp: "Formatted at 2026-08-07 18:06:04 on dist-test-slave-rnrw"
server_key: "e83d186b33d318f0fdb306e5fcaf3b6e"
server_key_iv: "2abed5a4e540ed2a0b5619552794a077"
server_key_version: "encryptionkey@0"
I20260807 18:06:04.590301 11658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260807 18:06:04.594201 11673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:04.595599 11658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:04.595765 11658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "a3341c3c6bcb4999bd311f33f8033f77"
format_stamp: "Formatted at 2026-08-07 18:06:04 on dist-test-slave-rnrw"
server_key: "e83d186b33d318f0fdb306e5fcaf3b6e"
server_key_iv: "2abed5a4e540ed2a0b5619552794a077"
server_key_version: "encryptionkey@0"
I20260807 18:06:04.595937 11658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:04.615648 11658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:04.620050 11658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:04.620347 11658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:04.640161 11658 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:36625
I20260807 18:06:04.640192 11726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:36625 every 8 connection(s)
I20260807 18:06:04.641654 11658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:04.645897 11727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:04.651618 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11658
I20260807 18:06:04.651854 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:04.652505 8442 external_mini_cluster.cc:1428] Setting key c217324119f932dad7992ccfd6851144
I20260807 18:06:04.658481 11727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77: Bootstrap starting.
I20260807 18:06:04.662057 11727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:04.663203 11727 log.cc:824] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:04.666314 11727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77: No bootstrap required, opened a new log
Aug 07 18:06:04 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:04.675105 11727 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } }
I20260807 18:06:04.675379 11730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:04.654787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49500 (local address 127.8.62.190:36625)
0807 18:06:04.655741 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:04.655754 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:04.655798 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:06:04.657591 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:04.657616 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:04.658038 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:04.658450 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:04.659882 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:04.661277 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:04.662577 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:04.663258 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:04.663461 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:04.667574 (+ 4113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:04.667581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:04.667596 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:04.667634 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:04.671262 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:04.671951 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:04.671967 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:04.671977 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:04.672155 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:04.672588 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:04.672595 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:04.672602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:04.673161 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:04.673506 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:04.673952 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 18:06:04.674396 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":450,"threads_started":1}
I20260807 18:06:04.675518 11727 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:04.675668 11727 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3341c3c6bcb4999bd311f33f8033f77, State: Initialized, Role: FOLLOWER
I20260807 18:06:04.676345 11727 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } }
I20260807 18:06:04.676637 11727 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:04.676789 11727 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:04.676987 11727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:04.679519 11727 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } }
I20260807 18:06:04.680115 11727 leader_election.cc:302] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3341c3c6bcb4999bd311f33f8033f77; no voters:
I20260807 18:06:04.680816 11727 leader_election.cc:288] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:04.680934 11732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:04.681643 11732 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [term 1 LEADER]: Becoming Leader. State: Replica: a3341c3c6bcb4999bd311f33f8033f77, State: Running, Role: LEADER
I20260807 18:06:04.682255 11732 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } }
I20260807 18:06:04.683133 11727 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:04.685406 11734 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3341c3c6bcb4999bd311f33f8033f77. Latest consensus state: current_term: 1 leader_uuid: "a3341c3c6bcb4999bd311f33f8033f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } } }
I20260807 18:06:04.685763 11734 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:04.690730 11733 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3341c3c6bcb4999bd311f33f8033f77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3341c3c6bcb4999bd311f33f8033f77" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36625 } } }
I20260807 18:06:04.691035 11739 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:04.691242 11733 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:04.696344 11739 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:04.705381 11739 catalog_manager.cc:1347] Generated new cluster ID: fd0ed6ca4bdc4348a85e91ec472e9279
I20260807 18:06:04.705514 11739 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:04.725646 11739 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:04.727640 11739 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:04.746031 11739 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3341c3c6bcb4999bd311f33f8033f77: Generated new TSK 0
I20260807 18:06:04.746948 11739 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:06:04.813194 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:39643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:05.159621 11753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:05.159952 11753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:05.160101 11753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:05.168080 11753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:05.168217 11753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:05.168416 11753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:05.180162 11753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:05.181635 11753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:05.183139 11753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:05.197592 11758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:05.197546 11761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:05.198102 11759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:05.198621 11753 server_base.cc:1034] running on GCE node
I20260807 18:06:05.199319 11753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:05.200533 11753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:05.201814 11753 hybrid_clock.cc:648] HybridClock initialized: now 1786125965201761 us; error 44 us; skew 500 ppm
I20260807 18:06:05.202196 11753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:05 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125965, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:05.209167 11753 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:06:05.210871 11753 webserver.cc:460] Webserver started at http://127.8.62.129:41407/ using document root <none> and password file <none>
I20260807 18:06:05.211696 11753 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:05.211869 11753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:05.212235 11753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:05.215641 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "8acbfc47487c4e2dac33bb99e257a5f6"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "876e597cbf11a3935e8863ff910437ae"
server_key_iv: "a55e750499b48bcd4f2df3f7a29df1a4"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.216606 11753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "8acbfc47487c4e2dac33bb99e257a5f6"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "876e597cbf11a3935e8863ff910437ae"
server_key_iv: "a55e750499b48bcd4f2df3f7a29df1a4"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.222648 11753 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20260807 18:06:05.226572 11768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.228018 11753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:06:05.228217 11753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "8acbfc47487c4e2dac33bb99e257a5f6"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "876e597cbf11a3935e8863ff910437ae"
server_key_iv: "a55e750499b48bcd4f2df3f7a29df1a4"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.228437 11753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:05.252224 11753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:05.256891 11753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:05.257237 11753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:05.258244 11753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:05.259634 11753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:05.259738 11753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.259883 11753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:05.259958 11753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.294317 11753 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:43163
I20260807 18:06:05.294379 11882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:43163 every 8 connection(s)
I20260807 18:06:05.295766 11753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:05.304427 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11753
I20260807 18:06:05.304697 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:05.305380 8442 external_mini_cluster.cc:1428] Setting key ad447356953b89b974a249d5bb2e1d84
Aug 07 18:06:05 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125965, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:05.317726 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:05.298511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:46245 (local address 127.8.62.190:36625)
0807 18:06:05.298974 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:05.298978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:05.300108 (+ 1130us) server_negotiation.cc:407] Connection header received
0807 18:06:05.302178 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:05.302184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:05.302282 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:05.302405 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:05.304037 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.304910 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.306698 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.307149 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.307304 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:05.311696 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:05.311702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:05.311705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:05.311739 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:05.314441 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:05.315270 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:05.315281 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:05.315283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:05.315406 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:05.315865 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:05.315870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:05.315873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:05.316146 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:05.316310 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:05.317096 (+ 786us) server_negotiation.cc:299] Negotiation successful
0807 18:06:05.317340 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":220,"threads_started":1}
I20260807 18:06:05.318753 11886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:05.299162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36625 (local address 127.8.62.129:46245)
0807 18:06:05.299951 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:05.299989 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:05.301873 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:05.302663 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:05.302673 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:05.303247 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:05.303851 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:05.303870 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.305089 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.305094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:05.306527 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:05.306540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.307402 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.307407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:05.307745 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:05.308536 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:05.308596 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:05.311414 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:05.314604 (+ 3190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:05.314614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:05.314634 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:05.315106 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:05.315530 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:05.315535 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:05.315539 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:05.315744 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:05.316288 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:05.316296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:05.316714 (+ 418us) client_negotiation.cc:769] Sending connection context
0807 18:06:05.316997 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 18:06:05.317578 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":320,"threads_started":1}
I20260807 18:06:05.320676 11883 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36625
I20260807 18:06:05.321005 11883 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:05.321817 11883 heartbeater.cc:511] Master 127.8.62.190:36625 requested a full tablet report, sending...
I20260807 18:06:05.324455 11691 ts_manager.cc:194] Registered new tserver with Master: 8acbfc47487c4e2dac33bb99e257a5f6 (127.8.62.129:43163)
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:05.326671 11691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:46245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:06:05.357115 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:39643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:05.704267 11889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:05.704629 11889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:05.704828 11889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:05.712184 11889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:05.712306 11889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:05.712421 11889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:05.722157 11889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:05.723421 11889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:05.724701 11889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:05.738773 11894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:05.738629 11895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:05.738730 11897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:05.739620 11889 server_base.cc:1034] running on GCE node
I20260807 18:06:05.740322 11889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:05.741402 11889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:05.742646 11889 hybrid_clock.cc:648] HybridClock initialized: now 1786125965742584 us; error 50 us; skew 500 ppm
I20260807 18:06:05.743043 11889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:05 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125965, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:05.746563 11889 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:05.748163 11889 webserver.cc:460] Webserver started at http://127.8.62.130:42367/ using document root <none> and password file <none>
I20260807 18:06:05.748916 11889 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:05.749024 11889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:05.749298 11889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:05.752233 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "7ff0049154fb4984afd1868e622e9793"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "d50b2402a375f365f7e3f4ea76332972"
server_key_iv: "e2f061abae61a92a93c0e9c614e8db28"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.753051 11889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "7ff0049154fb4984afd1868e622e9793"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "d50b2402a375f365f7e3f4ea76332972"
server_key_iv: "e2f061abae61a92a93c0e9c614e8db28"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.758471 11889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:06:05.762095 11904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.763566 11889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:06:05.763767 11889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "7ff0049154fb4984afd1868e622e9793"
format_stamp: "Formatted at 2026-08-07 18:06:05 on dist-test-slave-rnrw"
server_key: "d50b2402a375f365f7e3f4ea76332972"
server_key_iv: "e2f061abae61a92a93c0e9c614e8db28"
server_key_version: "encryptionkey@0"
I20260807 18:06:05.763978 11889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:05.783689 11889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:05.788368 11889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:05.788710 11889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:05.789719 11889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:05.791096 11889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:05.791180 11889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.791357 11889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:05.791409 11889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:05.824038 11889 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:38321
I20260807 18:06:05.824086 12019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:38321 every 8 connection(s)
I20260807 18:06:05.825430 11889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:05.828610 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 11889
I20260807 18:06:05.828835 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:05.829290 8442 external_mini_cluster.cc:1428] Setting key ff210e28895fd94fddc9dec05c190358
Aug 07 18:06:05 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125965, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:05.847648 12022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:05.828074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:60047 (local address 127.8.62.190:36625)
0807 18:06:05.828630 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:05.828637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:05.829904 (+ 1267us) server_negotiation.cc:407] Connection header received
0807 18:06:05.832100 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:05.832107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:05.832210 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:05.832328 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:05.833930 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.835073 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.836681 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.837186 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.837284 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:05.841604 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:05.841609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:05.841612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:05.841675 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:05.844356 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:05.845270 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:05.845282 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:05.845285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:05.845408 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:05.845882 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:05.845886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:05.845888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:05.846155 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:05.846326 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:05.847019 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 18:06:05.847247 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":254,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:06:05.848750 12023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:05.828798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36625 (local address 127.8.62.130:60047)
0807 18:06:05.829722 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:05.829776 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:05.831809 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:05.832640 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:05.832648 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:05.833182 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:05.833746 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:05.833763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.835231 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.835234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:05.836467 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:05.836479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:05.837432 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:05.837441 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:05.837774 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:05.838545 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:05.838586 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:05.841352 (+ 2766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:05.844507 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:05.844517 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:05.844535 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:05.845130 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:05.845526 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:05.845531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:05.845535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:05.845729 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:05.846287 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:05.846295 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:05.846655 (+ 360us) client_negotiation.cc:769] Sending connection context
0807 18:06:05.846922 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 18:06:05.847514 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":394,"threads_started":1}
I20260807 18:06:05.850378 12020 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36625
I20260807 18:06:05.850759 12020 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:06:05.851670 12020 heartbeater.cc:511] Master 127.8.62.190:36625 requested a full tablet report, sending...
I20260807 18:06:05.853348 11691 ts_manager.cc:194] Registered new tserver with Master: 7ff0049154fb4984afd1868e622e9793 (127.8.62.130:38321)
I20260807 18:06:05.854624 11691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:60047
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:05.877291 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:39643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:06.240785 12026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:06.241137 12026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:06.241317 12026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:06.249243 12026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:06.249400 12026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:06.249600 12026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:06.260656 12026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:36625
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:06.262276 12026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:06.263731 12026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:06.281000 12031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:06.281478 12032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:06.282197 12034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:06.282325 12026 server_base.cc:1034] running on GCE node
I20260807 18:06:06.283068 12026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:06.284225 12026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:06.285527 12026 hybrid_clock.cc:648] HybridClock initialized: now 1786125966285427 us; error 91 us; skew 500 ppm
I20260807 18:06:06.285917 12026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:06.289779 12026 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:06.291549 12026 webserver.cc:460] Webserver started at http://127.8.62.131:36727/ using document root <none> and password file <none>
I20260807 18:06:06.292308 12026 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:06.292464 12026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:06.292843 12026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:06.296007 12026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "eea7cadae6fa4c149592f95dabae44b4"
format_stamp: "Formatted at 2026-08-07 18:06:06 on dist-test-slave-rnrw"
server_key: "72c26e7c2250a4db2afc91d267bd379d"
server_key_iv: "66f9ac80571e60140bf19d28b365271c"
server_key_version: "encryptionkey@0"
I20260807 18:06:06.296911 12026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "eea7cadae6fa4c149592f95dabae44b4"
format_stamp: "Formatted at 2026-08-07 18:06:06 on dist-test-slave-rnrw"
server_key: "72c26e7c2250a4db2afc91d267bd379d"
server_key_iv: "66f9ac80571e60140bf19d28b365271c"
server_key_version: "encryptionkey@0"
I20260807 18:06:06.303231 12026 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:06:06.307206 12041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:06.309027 12026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:06.309226 12026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "eea7cadae6fa4c149592f95dabae44b4"
format_stamp: "Formatted at 2026-08-07 18:06:06 on dist-test-slave-rnrw"
server_key: "72c26e7c2250a4db2afc91d267bd379d"
server_key_iv: "66f9ac80571e60140bf19d28b365271c"
server_key_version: "encryptionkey@0"
I20260807 18:06:06.309439 12026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:06.322146 12026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:06.326838 12026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:06.327184 12026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:06.328327 12026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:06.329823 12026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:06.329927 12026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:06.330092 12026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:06.330165 12026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:06.330317 11883 heartbeater.cc:503] Master 127.8.62.190:36625 was elected leader, sending a full tablet report...
I20260807 18:06:06.380875 12026 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:46497
I20260807 18:06:06.380913 12153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:46497 every 8 connection(s)
I20260807 18:06:06.382306 12026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:06.389114 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12026
I20260807 18:06:06.389334 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:06.389811 8442 external_mini_cluster.cc:1428] Setting key 58e84456087a8ef100d6bbf84d971db7
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:06.403797 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.385332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:51243 (local address 127.8.62.190:36625)
0807 18:06:06.385794 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:06.385798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:06.387011 (+ 1213us) server_negotiation.cc:407] Connection header received
0807 18:06:06.389020 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:06.389026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:06.389123 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:06.389256 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:06.390916 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.391819 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.394007 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.394410 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.394519 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.397901 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:06.397906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:06.397909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:06.397942 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:06.401030 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.401692 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.401696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.401698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.401812 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.402182 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.402186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.402188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.402517 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:06.402668 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:06.403334 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 18:06:06.403501 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":219,"threads_started":1}
I20260807 18:06:06.405207 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.386024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:36625 (local address 127.8.62.131:51243)
0807 18:06:06.386836 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:06.386885 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:06.388733 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:06.389540 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:06.389549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:06.390126 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:06.390715 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:06.390758 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.391978 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.391982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:06.393837 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:06.393855 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.394623 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.394628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:06.394833 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.395460 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:06.395483 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:06.397704 (+ 2221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:06.401152 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:06.401158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:06.401170 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:06.401569 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:06.401901 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:06.401904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:06.401906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:06.402078 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:06.402671 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:06.402677 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:06.402983 (+ 306us) client_negotiation.cc:769] Sending connection context
0807 18:06:06.403205 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 18:06:06.403760 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":378,"threads_started":1}
I20260807 18:06:06.406719 12154 heartbeater.cc:347] Connected to a master server at 127.8.62.190:36625
I20260807 18:06:06.407001 12154 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:06.407698 12154 heartbeater.cc:511] Master 127.8.62.190:36625 requested a full tablet report, sending...
I20260807 18:06:06.409061 11691 ts_manager.cc:194] Registered new tserver with Master: eea7cadae6fa4c149592f95dabae44b4 (127.8.62.131:46497)
I20260807 18:06:06.409814 11691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:51243
I20260807 18:06:06.418411 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:06.471303 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.455797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49512 (local address 127.8.62.190:36625)
0807 18:06:06.456052 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:06.456059 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:06.456327 (+ 268us) server_negotiation.cc:407] Connection header received
0807 18:06:06.456491 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:06.456495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:06.456677 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:06.456769 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:06.458284 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.459012 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.460343 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.460836 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.460931 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.465752 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:06.465757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:06.465760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:06.465798 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:06.468816 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.469488 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.469492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.469494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.469621 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.470002 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.470005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.470007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.470336 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:06.470516 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:06.470792 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 18:06:06.470966 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 18:06:06.471920 11691 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49512
I20260807 18:06:06.473906 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11753
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:06.490350 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.475035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.8.62.129:43163)
0807 18:06:06.475711 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:06.475718 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:06.475743 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:06:06.475805 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:06.475811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:06.476015 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:06.476153 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:06.477291 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.478298 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.479432 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.480054 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.480154 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.484243 (+ 4089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:06.484256 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:06.484272 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:06.484306 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:06.487445 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.488270 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.488277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.488279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.488411 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.488882 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.488885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.488888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.489234 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:06.489461 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:06.489736 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 18:06:06.489935 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":381,"threads_started":1}
W20260807 18:06:06.491624 11878 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59068
W20260807 18:06:06.492786 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11753: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:06:06.493005 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11753
I20260807 18:06:06.510337 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11889
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:06:06.526209 12167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.511304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40298 (local address 127.8.62.130:38321)
0807 18:06:06.511928 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:06.511935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:06.511962 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:06:06.512032 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:06.512038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:06.512258 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:06.512398 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:06.513745 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.514922 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.516125 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.516759 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.516865 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.520892 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:06.520903 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:06.520918 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:06.520957 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:06.523743 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.524360 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.524367 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.524369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.524498 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.524873 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.524876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.524878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.525213 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:06.525446 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:06.525725 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 18:06:06.525921 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":344,"threads_started":1}
W20260807 18:06:06.527663 12015 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40298
W20260807 18:06:06.528869 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11889: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:06:06.529012 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11889
I20260807 18:06:06.545863 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12026
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[11645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125966, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:06:06.563051 12168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:06.547018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36254 (local address 127.8.62.131:46497)
0807 18:06:06.547731 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:06.547738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:06.547768 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:06:06.547830 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:06.547837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:06.548060 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:06.548254 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:06.549711 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.550759 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.552051 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:06.552708 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:06.552848 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:06.557200 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:06.557212 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:06.557227 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:06.557261 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:06.559970 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.560891 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.560901 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.560907 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.561036 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:06.561435 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:06.561439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:06.561441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:06.561786 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:06.562106 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:06.562406 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 18:06:06.562704 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":353,"threads_started":1}
W20260807 18:06:06.564365 12149 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36254
W20260807 18:06:06.565514 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12026: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:06:06.565682 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12026
I20260807 18:06:06.582513 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 11658
I20260807 18:06:06.583791 11721 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:06.690335 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 11658
2026-08-07T18:06:06Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3832 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[12172](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[12172](info): set up 2 sockets
Aug 07 18:06:06 dist-test-slave-rnrw krb5kdc[12172](info): commencing operation
krb5kdc: starting...
W20260807 18:06:07.807591 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:07 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:08.016877 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43161
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:43161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:08.352038 12185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:08.352372 12185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:08.352567 12185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:08.360176 12185 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:08.360313 12185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:08.360410 12185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:08.360504 12185 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:08.360571 12185 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:08.371078 12185 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:43161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:08.372610 12185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:08.374109 12185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:08.387773 12194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:08.387814 12191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:08.387740 12192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:08.388675 12185 server_base.cc:1034] running on GCE node
I20260807 18:06:08.389649 12185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:08.391229 12185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:08.392480 12185 hybrid_clock.cc:648] HybridClock initialized: now 1786125968392403 us; error 71 us; skew 500 ppm
I20260807 18:06:08.392992 12185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:08 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125968, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:08.396993 12185 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:08.398851 12185 webserver.cc:460] Webserver started at http://127.8.62.190:33985/ using document root <none> and password file <none>
I20260807 18:06:08.399629 12185 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:08.399784 12185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:08.400132 12185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:08.403354 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "0581efbb4a734220b3b04e8f9d8c20bf"
format_stamp: "Formatted at 2026-08-07 18:06:08 on dist-test-slave-rnrw"
server_key: "d3c93e5abfb5298436fb65a8e311c4ea"
server_key_iv: "82b70d7aa7884f177b906ddf0a86f53f"
server_key_version: "encryptionkey@0"
I20260807 18:06:08.404183 12185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "0581efbb4a734220b3b04e8f9d8c20bf"
format_stamp: "Formatted at 2026-08-07 18:06:08 on dist-test-slave-rnrw"
server_key: "d3c93e5abfb5298436fb65a8e311c4ea"
server_key_iv: "82b70d7aa7884f177b906ddf0a86f53f"
server_key_version: "encryptionkey@0"
I20260807 18:06:08.409981 12185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:06:08.413884 12201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:08.415396 12185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:08.415609 12185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "0581efbb4a734220b3b04e8f9d8c20bf"
format_stamp: "Formatted at 2026-08-07 18:06:08 on dist-test-slave-rnrw"
server_key: "d3c93e5abfb5298436fb65a8e311c4ea"
server_key_iv: "82b70d7aa7884f177b906ddf0a86f53f"
server_key_version: "encryptionkey@0"
I20260807 18:06:08.415834 12185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:08.444625 12185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:08.449599 12185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:08.449924 12185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:08.469429 12185 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:43161
I20260807 18:06:08.469465 12253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:43161 every 8 connection(s)
I20260807 18:06:08.470917 12185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:08.475425 12254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:08.477912 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12185
I20260807 18:06:08.478160 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:08.478987 8442 external_mini_cluster.cc:1428] Setting key f9e31470959f03ae1cd14f82c93beec0
I20260807 18:06:08.488308 12254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf: Bootstrap starting.
I20260807 18:06:08.491775 12254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:08.492905 12254 log.cc:824] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:06:08 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:08.496353 12254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf: No bootstrap required, opened a new log
I20260807 18:06:08.502964 12257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:08.481477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.8.62.190:43161)
0807 18:06:08.482324 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:08.482337 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:08.482383 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:06:08.484163 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:08.484190 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:08.484706 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:08.485152 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:08.486724 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:08.488089 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:08.489462 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:08.490189 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:08.490407 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:08.494914 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:08.494922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:08.494938 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:08.494979 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:08.498253 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:08.499222 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:08.499229 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:08.499240 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:08.499437 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:08.499898 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:08.499903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:08.499905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:08.500533 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:08.500840 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:08.501335 (+ 495us) server_negotiation.cc:299] Negotiation successful
0807 18:06:08.501803 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":354,"threads_started":1}
I20260807 18:06:08.505650 12254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } }
I20260807 18:06:08.506064 12254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:08.506203 12254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0581efbb4a734220b3b04e8f9d8c20bf, State: Initialized, Role: FOLLOWER
I20260807 18:06:08.506876 12254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } }
I20260807 18:06:08.507135 12254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:08.507285 12254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:08.507488 12254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:08.509843 12254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } }
I20260807 18:06:08.510378 12254 leader_election.cc:302] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0581efbb4a734220b3b04e8f9d8c20bf; no voters:
I20260807 18:06:08.511050 12254 leader_election.cc:288] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:08.511209 12259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:08.512460 12259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [term 1 LEADER]: Becoming Leader. State: Replica: 0581efbb4a734220b3b04e8f9d8c20bf, State: Running, Role: LEADER
I20260807 18:06:08.513207 12259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } }
I20260807 18:06:08.513490 12254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:08.516085 12261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0581efbb4a734220b3b04e8f9d8c20bf. Latest consensus state: current_term: 1 leader_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } } }
I20260807 18:06:08.516464 12261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:08.517169 12260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0581efbb4a734220b3b04e8f9d8c20bf" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43161 } } }
I20260807 18:06:08.517539 12260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:08.520334 12266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:08.525681 12266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:08.535604 12266 catalog_manager.cc:1347] Generated new cluster ID: 8290e5695bd34caf913114ef7f177b9d
I20260807 18:06:08.535737 12266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:08.558821 12266 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:08.560310 12266 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:08.567767 12266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0581efbb4a734220b3b04e8f9d8c20bf: Generated new TSK 0
I20260807 18:06:08.568768 12266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:06:08.634665 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:08.979525 12281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:08.979832 12281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:08.979926 12281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:08.987550 12281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:08.987671 12281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:08.987788 12281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:08.998549 12281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:08.999835 12281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:09.001272 12281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:09.015597 12286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:09.015401 12287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:09.016127 12289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:09.016729 12281 server_base.cc:1034] running on GCE node
I20260807 18:06:09.017465 12281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:09.018482 12281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:09.019734 12281 hybrid_clock.cc:648] HybridClock initialized: now 1786125969019669 us; error 54 us; skew 500 ppm
I20260807 18:06:09.020124 12281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:09 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:09.023641 12281 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:06:09.025283 12281 webserver.cc:460] Webserver started at http://127.8.62.129:38453/ using document root <none> and password file <none>
I20260807 18:06:09.026031 12281 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:09.026177 12281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:09.026511 12281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:09.029687 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "048e230381e248f499c22870bdcc4a7d"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "b6e247b34d7c292d3b4a47427fd1d72f"
server_key_iv: "0071a3d31a1a09a5c12d1f7eb93347f0"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.030496 12281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "048e230381e248f499c22870bdcc4a7d"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "b6e247b34d7c292d3b4a47427fd1d72f"
server_key_iv: "0071a3d31a1a09a5c12d1f7eb93347f0"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.036130 12281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:09.039979 12296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:09.041502 12281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:09.041702 12281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "048e230381e248f499c22870bdcc4a7d"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "b6e247b34d7c292d3b4a47427fd1d72f"
server_key_iv: "0071a3d31a1a09a5c12d1f7eb93347f0"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.041916 12281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:09.067318 12281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:09.072053 12281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:09.072382 12281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:09.073499 12281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:09.075013 12281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:09.075091 12281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:09.075256 12281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:09.075309 12281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:09.112337 12281 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:44465
I20260807 18:06:09.112481 12410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:44465 every 8 connection(s)
I20260807 18:06:09.113855 12281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:09.116844 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12281
I20260807 18:06:09.117136 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:09.117939 8442 external_mini_cluster.cc:1428] Setting key 9cc86d996756030711606d6855fbfd05
Aug 07 18:06:09 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:09.139328 12413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:09.116873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:45411 (local address 127.8.62.190:43161)
0807 18:06:09.117418 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:09.117426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:09.118815 (+ 1389us) server_negotiation.cc:407] Connection header received
0807 18:06:09.121009 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:09.121017 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:09.121111 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:09.121228 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:09.122820 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.124016 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.125723 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.126253 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.126363 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:09.130872 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:09.130883 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:09.130893 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:09.130948 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:09.135243 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:09.136159 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:09.136169 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:09.136176 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:09.136366 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:09.137097 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:09.137101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:09.137103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:09.137451 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:09.137583 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:09.138823 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0807 18:06:09.139042 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":254,"threads_started":1}
I20260807 18:06:09.141281 12414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:09.117599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43161 (local address 127.8.62.129:45411)
0807 18:06:09.118582 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:09.118630 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:09.120751 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:09.121451 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:09.121459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:09.122035 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:09.122631 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:09.122650 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.124183 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.124187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:09.125564 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:09.125577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.126573 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.126585 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:09.126927 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:09.127833 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:09.127858 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:09.130570 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:09.135433 (+ 4863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:09.135442 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:09.135458 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:09.135979 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:09.136581 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:09.136590 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:09.136596 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:09.136924 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:09.137637 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:09.137649 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:09.138273 (+ 624us) client_negotiation.cc:769] Sending connection context
0807 18:06:09.138692 (+ 419us) client_negotiation.cc:240] Negotiation successful
0807 18:06:09.139409 (+ 717us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":393,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:09.143209 12411 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43161
I20260807 18:06:09.143570 12411 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:09.144383 12411 heartbeater.cc:511] Master 127.8.62.190:43161 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:06:09.146888 12219 ts_manager.cc:194] Registered new tserver with Master: 048e230381e248f499c22870bdcc4a7d (127.8.62.129:44465)
I20260807 18:06:09.148468 12219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:45411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:06:09.173375 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:09.511147 12417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:09.511439 12417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:09.511621 12417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:09.519266 12417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:09.519389 12417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:09.519605 12417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:09.529552 12417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:09.530956 12417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:09.532315 12417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:09.546415 12424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:09.546474 12426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:09.546444 12423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:09.548151 12417 server_base.cc:1034] running on GCE node
I20260807 18:06:09.548907 12417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:09.549980 12417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:09.551256 12417 hybrid_clock.cc:648] HybridClock initialized: now 1786125969551213 us; error 53 us; skew 500 ppm
I20260807 18:06:09.551707 12417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:09 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:09.555089 12417 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:09.556659 12417 webserver.cc:460] Webserver started at http://127.8.62.130:33589/ using document root <none> and password file <none>
I20260807 18:06:09.557387 12417 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:09.557539 12417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:09.557912 12417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:09.561218 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "d9b19fd3ad684b8e94d4809cbb32595c"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "1b6e5b398afeb3dc973be1bbe163db9f"
server_key_iv: "4d7064b85acca7b4483f86c6d0839bab"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.562111 12417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "d9b19fd3ad684b8e94d4809cbb32595c"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "1b6e5b398afeb3dc973be1bbe163db9f"
server_key_iv: "4d7064b85acca7b4483f86c6d0839bab"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.567668 12417 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:06:09.571358 12433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:09.572933 12417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:06:09.573122 12417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "d9b19fd3ad684b8e94d4809cbb32595c"
format_stamp: "Formatted at 2026-08-07 18:06:09 on dist-test-slave-rnrw"
server_key: "1b6e5b398afeb3dc973be1bbe163db9f"
server_key_iv: "4d7064b85acca7b4483f86c6d0839bab"
server_key_version: "encryptionkey@0"
I20260807 18:06:09.573348 12417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:09.596909 12417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:09.601517 12417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:09.601856 12417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:09.602896 12417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:09.604290 12417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:09.604365 12417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:09.604525 12417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:09.604596 12417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 18:06:09.640986 12417 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:37141
I20260807 18:06:09.641068 12547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:37141 every 8 connection(s)
I20260807 18:06:09.642426 12417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:09.645296 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12417
I20260807 18:06:09.645509 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:09.645967 8442 external_mini_cluster.cc:1428] Setting key 31447113a0d499f6bd11cb91cb49f1b5
Aug 07 18:06:09 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125969, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:06:09.665328 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:09.646752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:46887 (local address 127.8.62.190:43161)
0807 18:06:09.647855 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:09.647862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:09.647882 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:06:09.649485 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:09.649490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:09.649592 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:09.649717 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:09.651290 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.652054 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.654328 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.654744 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.654843 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:09.659194 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:09.659199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:09.659202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:09.659237 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:09.662134 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:09.662918 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:09.662923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:09.662925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:09.663059 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:09.663522 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:09.663526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:09.663528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:09.663792 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:09.663948 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:09.664700 (+ 752us) server_negotiation.cc:299] Negotiation successful
0807 18:06:09.664937 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":259,"threads_started":1}
I20260807 18:06:09.666347 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:09.645833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43161 (local address 127.8.62.130:46887)
0807 18:06:09.647435 (+ 1602us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:09.647481 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:09.649208 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:09.649972 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:09.649981 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:09.650528 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:09.651109 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:09.651128 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.652221 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.652225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:09.654141 (+ 1916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:09.654166 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:09.654980 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:09.654989 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:09.655339 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:09.656118 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:09.656154 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:09.658924 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:09.662287 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:09.662296 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:09.662316 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:09.662772 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:09.663176 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:09.663181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:09.663185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:09.663382 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:09.663937 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:09.663945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:09.664304 (+ 359us) client_negotiation.cc:769] Sending connection context
0807 18:06:09.664597 (+ 293us) client_negotiation.cc:240] Negotiation successful
0807 18:06:09.665187 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"thread_start_us":486,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:06:09.667989 12548 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43161
I20260807 18:06:09.668613 12548 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:09.669422 12548 heartbeater.cc:511] Master 127.8.62.190:43161 requested a full tablet report, sending...
I20260807 18:06:09.671038 12219 ts_manager.cc:194] Registered new tserver with Master: d9b19fd3ad684b8e94d4809cbb32595c (127.8.62.130:37141)
I20260807 18:06:09.671921 12219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:46887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:09.692310 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:10.025770 12555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:10.026054 12555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:10.026208 12555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:10.033658 12555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:10.033818 12555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:10.034039 12555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:10.044175 12555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:10.045583 12555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:10.046912 12555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:10.060701 12561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:10.060799 12560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:10.060783 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:10.061769 12555 server_base.cc:1034] running on GCE node
I20260807 18:06:10.062464 12555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:10.063588 12555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:10.064843 12555 hybrid_clock.cc:648] HybridClock initialized: now 1786125970064784 us; error 77 us; skew 500 ppm
I20260807 18:06:10.065217 12555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:10.068658 12555 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:10.070176 12555 webserver.cc:460] Webserver started at http://127.8.62.131:43489/ using document root <none> and password file <none>
I20260807 18:06:10.071031 12555 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:10.071188 12555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:10.071523 12555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:10.074630 12555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "c5b66bf5a71a44958dfc716050d30754"
format_stamp: "Formatted at 2026-08-07 18:06:10 on dist-test-slave-rnrw"
server_key: "956a8cd37cb3d594f567dcd399e98d03"
server_key_iv: "3be0e33e62fb2a012d47c495b5dcb068"
server_key_version: "encryptionkey@0"
I20260807 18:06:10.075443 12555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "c5b66bf5a71a44958dfc716050d30754"
format_stamp: "Formatted at 2026-08-07 18:06:10 on dist-test-slave-rnrw"
server_key: "956a8cd37cb3d594f567dcd399e98d03"
server_key_iv: "3be0e33e62fb2a012d47c495b5dcb068"
server_key_version: "encryptionkey@0"
I20260807 18:06:10.081199 12555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:06:10.085040 12570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:10.086570 12555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:10.086761 12555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "c5b66bf5a71a44958dfc716050d30754"
format_stamp: "Formatted at 2026-08-07 18:06:10 on dist-test-slave-rnrw"
server_key: "956a8cd37cb3d594f567dcd399e98d03"
server_key_iv: "3be0e33e62fb2a012d47c495b5dcb068"
server_key_version: "encryptionkey@0"
I20260807 18:06:10.086982 12555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:10.112277 12555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:10.117928 12555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:10.118248 12555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:10.119431 12555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:10.120923 12555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:10.121001 12555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:10.121084 12555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:10.121127 12555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:10.151949 12411 heartbeater.cc:503] Master 127.8.62.190:43161 was elected leader, sending a full tablet report...
I20260807 18:06:10.156232 12555 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:39183
I20260807 18:06:10.156286 12685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:39183 every 8 connection(s)
I20260807 18:06:10.157723 12555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:10.163538 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12555
I20260807 18:06:10.163779 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:10.164347 8442 external_mini_cluster.cc:1428] Setting key bf40a6f95699ffbedf4df6f9b3c3a729
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:10.182410 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.160595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:38187 (local address 127.8.62.190:43161)
0807 18:06:10.160815 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.160822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.162231 (+ 1409us) server_negotiation.cc:407] Connection header received
0807 18:06:10.165324 (+ 3093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.165329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.165429 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.165655 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.167964 (+ 2309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.168984 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.170660 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.171106 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.171219 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.175167 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.175173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.175176 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.175218 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.178142 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.179302 (+ 1160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.179309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.179312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.179475 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.180077 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.180082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.180084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.180520 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.180731 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.181880 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.182132 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:06:10.183604 12688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.161162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43161 (local address 127.8.62.131:38187)
0807 18:06:10.162046 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:10.162088 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:10.164656 (+ 2568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:10.165980 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:10.165996 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:10.166750 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:10.167709 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:10.167740 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.169155 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.169159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:10.170467 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:10.170481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.171324 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.171329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:10.171568 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.172377 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:10.172403 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:10.174899 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:10.178386 (+ 3487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:10.178398 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:10.178420 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:10.179163 (+ 743us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:10.179615 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:10.179620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:10.179625 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:10.179918 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:10.180747 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:10.180757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:10.181387 (+ 630us) client_negotiation.cc:769] Sending connection context
0807 18:06:10.181722 (+ 335us) client_negotiation.cc:240] Negotiation successful
0807 18:06:10.182222 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":390,"threads_started":1}
I20260807 18:06:10.185501 12686 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43161
I20260807 18:06:10.185945 12686 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:10.186811 12686 heartbeater.cc:511] Master 127.8.62.190:43161 requested a full tablet report, sending...
I20260807 18:06:10.188751 12219 ts_manager.cc:194] Registered new tserver with Master: c5b66bf5a71a44958dfc716050d30754 (127.8.62.131:39183)
I20260807 18:06:10.189764 12219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:38187
I20260807 18:06:10.194180 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:10.252123 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.232085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.8.62.129:44465)
0807 18:06:10.232904 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.232914 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.232949 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:06:10.233017 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.233032 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.233247 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.233440 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.234920 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.235967 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.237258 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.237831 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.237970 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.246175 (+ 8205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.246184 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.246199 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.246241 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.249450 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.250186 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.250194 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.250196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.250332 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.250683 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.250687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.250689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.251061 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.251289 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.251575 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.251802 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":493,"threads_started":1}
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:10.272162 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.257443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.8.62.190:43161)
0807 18:06:10.257696 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.257701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.257764 (+ 63us) server_negotiation.cc:407] Connection header received
0807 18:06:10.257918 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.257921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.258000 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.258097 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.259444 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.260197 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.261523 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.261965 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.262077 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.266349 (+ 4272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.266355 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.266358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.266402 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.269300 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.270120 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.270125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.270127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.270266 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.270705 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.270710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.270712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.271067 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.271278 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.271612 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.271793 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260807 18:06:10.272855 12219 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48576
I20260807 18:06:10.274055 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12281
I20260807 18:06:10.288110 12693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.275140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.8.62.129:44465)
0807 18:06:10.275383 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.275390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.275591 (+ 201us) server_negotiation.cc:407] Connection header received
0807 18:06:10.275788 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.275800 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.275908 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.276036 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.277574 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.278287 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.279513 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.279957 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.280119 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.282596 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.282603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.282606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.282643 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.285452 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.286158 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.286165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.286168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.286305 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.286682 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.286687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.286692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.286965 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.287218 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.287550 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.287780 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:06:10.288918 12406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:10.373786 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12281
I20260807 18:06:10.393183 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12417
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:06:10.410437 12698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.394250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.8.62.130:37141)
0807 18:06:10.394866 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.394873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.394900 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:06:10.394985 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.394992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.395210 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.395341 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.396708 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.397708 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.399032 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.399586 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.399686 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.404239 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.404254 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.404270 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.404321 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.407485 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.408326 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.408333 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.408336 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.408467 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.408958 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.408962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.408964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.409339 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.409588 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.409929 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.410137 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":370,"threads_started":1}
I20260807 18:06:10.411902 12543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:10.495683 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12417
I20260807 18:06:10.512804 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12555
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12172](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125970, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:06:10.530978 12700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:10.514019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.8.62.131:39183)
0807 18:06:10.514926 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:10.514934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:10.514971 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:06:10.515040 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:10.515046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:10.515272 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:10.515417 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:10.516879 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.518214 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.519718 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:10.520390 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:10.520521 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:10.524888 (+ 4367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:10.524899 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:10.524916 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:10.524951 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:10.527921 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.528910 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.528918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.528920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.529042 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:10.529444 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:10.529448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:10.529450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:10.529840 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:10.530091 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:10.530337 (+ 246us) server_negotiation.cc:299] Negotiation successful
0807 18:06:10.530606 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":562,"threads_started":1}
I20260807 18:06:10.532339 12681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:10.617365 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12555
I20260807 18:06:10.637243 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12185
I20260807 18:06:10.638509 12249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:10.742921 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12185
2026-08-07T18:06:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4054 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12705](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12705](info): set up 2 sockets
Aug 07 18:06:10 dist-test-slave-rnrw krb5kdc[12705](info): commencing operation
krb5kdc: starting...
W20260807 18:06:11.871861 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:12 dist-test-slave-rnrw krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:12.093577 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40825
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:40825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:12.436656 12718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:12.436981 12718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:12.437212 12718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:12.444752 12718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:12.444865 12718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:12.444980 12718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:12.445091 12718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:12.445166 12718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:12.455384 12718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:40825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:12.456871 12718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:12.458235 12718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:12.470081 12726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:12.470144 12724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:12.470301 12723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:12.471508 12718 server_base.cc:1034] running on GCE node
I20260807 18:06:12.472368 12718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:12.473913 12718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:12.475143 12718 hybrid_clock.cc:648] HybridClock initialized: now 1786125972475081 us; error 80 us; skew 500 ppm
I20260807 18:06:12.475521 12718 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:12 dist-test-slave-rnrw krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:12.479352 12718 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:12.480754 12718 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:12.480856 12718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:12.481223 12718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:12.484287 12718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "0b1c8e3594634deba4f1044cae8c4bc1"
format_stamp: "Formatted at 2026-08-07 18:06:12 on dist-test-slave-rnrw"
server_key: "15fdb332cb141a2d3c40c84b153fa90b"
server_key_iv: "c22db0cf23169f2534edc420d1d989cd"
server_key_version: "encryptionkey@0"
I20260807 18:06:12.485209 12718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "0b1c8e3594634deba4f1044cae8c4bc1"
format_stamp: "Formatted at 2026-08-07 18:06:12 on dist-test-slave-rnrw"
server_key: "15fdb332cb141a2d3c40c84b153fa90b"
server_key_iv: "c22db0cf23169f2534edc420d1d989cd"
server_key_version: "encryptionkey@0"
I20260807 18:06:12.490576 12718 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:06:12.494227 12732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:12.495868 12718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:06:12.496094 12718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "0b1c8e3594634deba4f1044cae8c4bc1"
format_stamp: "Formatted at 2026-08-07 18:06:12 on dist-test-slave-rnrw"
server_key: "15fdb332cb141a2d3c40c84b153fa90b"
server_key_iv: "c22db0cf23169f2534edc420d1d989cd"
server_key_version: "encryptionkey@0"
I20260807 18:06:12.496340 12718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:12.537684 12718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:12.542151 12718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:12.542449 12718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:12.561601 12718 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:40825
I20260807 18:06:12.561645 12786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:40825 every 8 connection(s)
I20260807 18:06:12.562521 12718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:12.566197 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12718
I20260807 18:06:12.566481 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:12.566507 12787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:12.567277 8442 external_mini_cluster.cc:1428] Setting key 3fd79918e13e3007166ae2613f158321
I20260807 18:06:12.579401 12787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: Bootstrap starting.
Aug 07 18:06:12 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:12.582847 12787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:12.584061 12787 log.cc:824] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:12.586932 12787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: No bootstrap required, opened a new log
I20260807 18:06:12.589560 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:12.569435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.8.62.190:40825)
0807 18:06:12.570278 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:12.570290 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:12.570335 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:06:12.571884 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:12.571905 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:12.572286 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:12.572718 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:12.574226 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:12.575514 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:12.576831 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:12.577550 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:12.577777 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:12.582177 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:12.582186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:12.582199 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:12.582240 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:12.585411 (+ 3171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:12.586122 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:12.586130 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:12.586136 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:12.586299 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:12.586690 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:12.586694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:12.586696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:12.587279 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:12.587621 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:12.588052 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 18:06:12.588479 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":336,"threads_started":1}
I20260807 18:06:12.594781 12787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } }
I20260807 18:06:12.595144 12787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:12.595268 12787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b1c8e3594634deba4f1044cae8c4bc1, State: Initialized, Role: FOLLOWER
I20260807 18:06:12.595889 12787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } }
I20260807 18:06:12.596098 12787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:12.596243 12787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:12.596421 12787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:12.598713 12787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } }
I20260807 18:06:12.599236 12787 leader_election.cc:302] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b1c8e3594634deba4f1044cae8c4bc1; no voters:
I20260807 18:06:12.599912 12787 leader_election.cc:288] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:12.600032 12792 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:12.601428 12792 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [term 1 LEADER]: Becoming Leader. State: Replica: 0b1c8e3594634deba4f1044cae8c4bc1, State: Running, Role: LEADER
I20260807 18:06:12.601990 12792 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } }
I20260807 18:06:12.602293 12787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:12.604846 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b1c8e3594634deba4f1044cae8c4bc1. Latest consensus state: current_term: 1 leader_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } } }
I20260807 18:06:12.605319 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:12.604899 12793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b1c8e3594634deba4f1044cae8c4bc1" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40825 } } }
I20260807 18:06:12.605821 12793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:12.610641 12799 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:12.615470 12799 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:12.624765 12799 catalog_manager.cc:1347] Generated new cluster ID: 98b93588c0e240aea3fb2bdd81827ba6
I20260807 18:06:12.624923 12799 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:12.637787 12799 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:12.639788 12799 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:12.656931 12799 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: Generated new TSK 0
I20260807 18:06:12.657878 12799 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:06:12.716333 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:13.058290 12813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:13.058583 12813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:13.058681 12813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:13.066171 12813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:13.066345 12813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:13.066484 12813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:13.076614 12813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:13.077939 12813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:13.079263 12813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:13.093035 12818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:13.093062 12821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:13.093171 12819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:13.094421 12813 server_base.cc:1034] running on GCE node
I20260807 18:06:13.095148 12813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:13.096211 12813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:13.097514 12813 hybrid_clock.cc:648] HybridClock initialized: now 1786125973097350 us; error 112 us; skew 500 ppm
I20260807 18:06:13.097892 12813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:13 dist-test-slave-rnrw krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125973, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:13.101655 12813 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:06:13.102965 12813 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:13.103116 12813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:13.103487 12813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:13.106660 12813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "cca2beb6317b7c2a53010aa0b17764be"
server_key_iv: "27a896d584f1f6caca90badbf284c5aa"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.107592 12813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "cca2beb6317b7c2a53010aa0b17764be"
server_key_iv: "27a896d584f1f6caca90badbf284c5aa"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.113511 12813 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:06:13.117340 12827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.118724 12813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:13.118906 12813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "cca2beb6317b7c2a53010aa0b17764be"
server_key_iv: "27a896d584f1f6caca90badbf284c5aa"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.119153 12813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:13.139812 12813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:13.144064 12813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:13.144361 12813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:13.145423 12813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:13.146878 12813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:13.146968 12813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.147147 12813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:13.147250 12813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.182085 12813 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:41145
I20260807 18:06:13.182140 12940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:41145 every 8 connection(s)
I20260807 18:06:13.183066 12813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:13.189164 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12813
I20260807 18:06:13.189452 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:13.190363 8442 external_mini_cluster.cc:1428] Setting key e688949c1b515600792b208a9b5d4e94
Aug 07 18:06:13 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125973, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:13.211280 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:13.186143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:45383 (local address 127.8.62.190:40825)
0807 18:06:13.186660 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:13.186666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:13.188014 (+ 1348us) server_negotiation.cc:407] Connection header received
0807 18:06:13.191137 (+ 3123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:13.191144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:13.191263 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:13.191398 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:13.193735 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.194543 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.197496 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.197919 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.198028 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:13.202577 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:13.202587 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:13.202596 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:13.202649 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:13.206803 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:13.207877 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:13.207886 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:13.207893 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:13.208062 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:13.208727 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:13.208735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:13.208741 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:13.209172 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:13.209337 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:13.210387 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0807 18:06:13.210703 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":238,"threads_started":1}
I20260807 18:06:13.212220 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:13.186773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40825 (local address 127.8.62.129:45383)
0807 18:06:13.187624 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:13.187818 (+ 194us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:13.190701 (+ 2883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:13.191725 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:13.191741 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:13.192479 (+ 738us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:13.193498 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:13.193532 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.194758 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.194768 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:13.197309 (+ 2541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:13.197331 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.198189 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.198199 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:13.198575 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:13.199416 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:13.199458 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:13.202275 (+ 2817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:13.207016 (+ 4741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:13.207026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:13.207044 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:13.207704 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:13.208199 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:13.208202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:13.208205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:13.208417 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:13.209319 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:13.209350 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:13.209875 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 18:06:13.210241 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 18:06:13.210835 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":334,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:13.214154 12941 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40825
I20260807 18:06:13.214550 12941 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:13.215524 12941 heartbeater.cc:511] Master 127.8.62.190:40825 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:06:13.218220 12749 ts_manager.cc:194] Registered new tserver with Master: fa97292aec3c4c9bbde724cb0e5e7cf9 (127.8.62.129:41145)
I20260807 18:06:13.220734 12749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:45383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:06:13.244525 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:13.580695 12947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:13.581058 12947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:13.581250 12947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:13.588747 12947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:13.588914 12947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:13.589097 12947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:13.599463 12947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:13.601084 12947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:13.602458 12947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:13.616854 12955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:13.616892 12953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:13.617069 12952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:13.617230 12947 server_base.cc:1034] running on GCE node
I20260807 18:06:13.618157 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:13.619167 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:13.620394 12947 hybrid_clock.cc:648] HybridClock initialized: now 1786125973620317 us; error 64 us; skew 500 ppm
I20260807 18:06:13.620925 12947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:13 dist-test-slave-rnrw krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125973, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:13.624871 12947 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:13.626149 12947 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:13.626281 12947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:13.626644 12947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:13.629865 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "34e5005cd00940a1a6c528f121b86913"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "41eb34509d91470f1304feed1d26758b"
server_key_iv: "55f33f940b47d547d0ff35bbb298cbf7"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.630729 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "34e5005cd00940a1a6c528f121b86913"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "41eb34509d91470f1304feed1d26758b"
server_key_iv: "55f33f940b47d547d0ff35bbb298cbf7"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.636343 12947 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:13.640194 12961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.641839 12947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:13.642046 12947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "34e5005cd00940a1a6c528f121b86913"
format_stamp: "Formatted at 2026-08-07 18:06:13 on dist-test-slave-rnrw"
server_key: "41eb34509d91470f1304feed1d26758b"
server_key_iv: "55f33f940b47d547d0ff35bbb298cbf7"
server_key_version: "encryptionkey@0"
I20260807 18:06:13.642288 12947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:13.661540 12947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:13.666244 12947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:13.666565 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:13.667579 12947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:13.669039 12947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:13.669163 12947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.669303 12947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:13.669392 12947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:13.704730 12947 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:39833
I20260807 18:06:13.704818 13076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:39833 every 8 connection(s)
I20260807 18:06:13.705686 12947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:13.715451 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 12947
I20260807 18:06:13.715667 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:13.716142 8442 external_mini_cluster.cc:1428] Setting key 6bc11e7ab7bb6d25392ed4c7370c5fa1
Aug 07 18:06:13 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125973, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:13.727012 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:13.708380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:46955 (local address 127.8.62.190:40825)
0807 18:06:13.708664 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:13.708669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:13.709878 (+ 1209us) server_negotiation.cc:407] Connection header received
0807 18:06:13.711792 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:13.711798 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:13.711896 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:13.712023 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:13.713671 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.714370 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.716045 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.716581 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.716723 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:13.720658 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:13.720663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:13.720666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:13.720700 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:13.723594 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:13.724513 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:13.724520 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:13.724524 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:13.724708 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:13.725220 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:13.725231 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:13.725234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:13.725512 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:13.725677 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:13.726409 (+ 732us) server_negotiation.cc:299] Negotiation successful
0807 18:06:13.726667 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 18:06:13.728058 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:13.708860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40825 (local address 127.8.62.130:46955)
0807 18:06:13.709715 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:13.709755 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:13.711512 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:13.712253 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:13.712261 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:13.712877 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:13.713477 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:13.713495 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.714544 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.714548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:13.715899 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:13.715911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:13.716814 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:13.716821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:13.717058 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:13.717742 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:13.717766 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:13.720353 (+ 2587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:13.723794 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:13.723811 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:13.723825 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:13.724383 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:13.724869 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:13.724875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:13.724878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:13.725090 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:13.725663 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:13.725672 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:13.726037 (+ 365us) client_negotiation.cc:769] Sending connection context
0807 18:06:13.726319 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 18:06:13.726888 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":319,"threads_started":1}
I20260807 18:06:13.729835 13077 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40825
I20260807 18:06:13.730196 13077 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:13.730911 13077 heartbeater.cc:511] Master 127.8.62.190:40825 requested a full tablet report, sending...
I20260807 18:06:13.732530 12749 ts_manager.cc:194] Registered new tserver with Master: 34e5005cd00940a1a6c528f121b86913 (127.8.62.130:39833)
I20260807 18:06:13.733927 12749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:46955
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:13.766346 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:14.109921 13082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:14.110229 13082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:14.110386 13082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:14.117954 13082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:14.118083 13082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:14.118263 13082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:14.128914 13082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:14.130343 13082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:14.131851 13082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:14.146600 13088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:14.146589 13087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:14.146667 13091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:14.147500 13082 server_base.cc:1034] running on GCE node
I20260807 18:06:14.148283 13082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:14.149433 13082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:14.150707 13082 hybrid_clock.cc:648] HybridClock initialized: now 1786125974150654 us; error 52 us; skew 500 ppm
I20260807 18:06:14.151120 13082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:14 dist-test-slave-rnrw krb5kdc[12705](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125974, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:14.154870 13082 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:14.156258 13082 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:14.156407 13082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:14.156838 13082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:14.160063 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "ff31a8dcd0ba470380c716720be7c82f"
format_stamp: "Formatted at 2026-08-07 18:06:14 on dist-test-slave-rnrw"
server_key: "66bae00cfa87ef13aa5c48d9317f6a1d"
server_key_iv: "f33f4cbb2f4bbf505887d891b44d2528"
server_key_version: "encryptionkey@0"
I20260807 18:06:14.161027 13082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "ff31a8dcd0ba470380c716720be7c82f"
format_stamp: "Formatted at 2026-08-07 18:06:14 on dist-test-slave-rnrw"
server_key: "66bae00cfa87ef13aa5c48d9317f6a1d"
server_key_iv: "f33f4cbb2f4bbf505887d891b44d2528"
server_key_version: "encryptionkey@0"
I20260807 18:06:14.167028 13082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:06:14.171372 13098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:14.173213 13082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:06:14.173439 13082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "ff31a8dcd0ba470380c716720be7c82f"
format_stamp: "Formatted at 2026-08-07 18:06:14 on dist-test-slave-rnrw"
server_key: "66bae00cfa87ef13aa5c48d9317f6a1d"
server_key_iv: "f33f4cbb2f4bbf505887d891b44d2528"
server_key_version: "encryptionkey@0"
I20260807 18:06:14.173704 13082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:14.194720 13082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:14.199370 13082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:14.199779 13082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:14.201007 13082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:14.202623 13082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:14.202749 13082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:14.202888 13082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:14.202977 13082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:14.224457 12941 heartbeater.cc:503] Master 127.8.62.190:40825 was elected leader, sending a full tablet report...
I20260807 18:06:14.238754 13082 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:42745
I20260807 18:06:14.238862 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:42745 every 8 connection(s)
I20260807 18:06:14.239794 13082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:14.248840 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13082
I20260807 18:06:14.249092 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:14.249578 8442 external_mini_cluster.cc:1428] Setting key 4c90ca26d0adc539807662f31b554037
Aug 07 18:06:14 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125974, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:14.260942 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.242482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:42187 (local address 127.8.62.190:40825)
0807 18:06:14.242961 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.242967 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.243960 (+ 993us) server_negotiation.cc:407] Connection header received
0807 18:06:14.245952 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.245957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.246032 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.246127 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:14.247796 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.248666 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.250575 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.251097 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.251227 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.254997 (+ 3770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:14.255004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:14.255019 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:14.255054 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:14.257730 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.258510 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.258515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.258519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.258653 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.259085 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.259091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.259094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.259379 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:14.259545 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.260267 (+ 722us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.260491 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
I20260807 18:06:14.261936 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.243066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40825 (local address 127.8.62.131:42187)
0807 18:06:14.243787 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.243837 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.245670 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.246327 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.246336 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.246986 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:14.247623 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.247640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.248815 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.248823 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.250415 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.250428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.251332 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.251339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.251577 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.252282 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:14.252305 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:14.254785 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:14.257892 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:14.257901 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:14.257920 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:14.258375 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:14.258768 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:14.258773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:14.258777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:14.258971 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:14.259532 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:14.259540 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:14.259892 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.260176 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.260786 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":290,"threads_started":1}
I20260807 18:06:14.263502 13214 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40825
I20260807 18:06:14.263813 13214 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:14.264461 13214 heartbeater.cc:511] Master 127.8.62.190:40825 requested a full tablet report, sending...
I20260807 18:06:14.266063 12749 ts_manager.cc:194] Registered new tserver with Master: ff31a8dcd0ba470380c716720be7c82f (127.8.62.131:42745)
I20260807 18:06:14.267171 12749 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:42187
I20260807 18:06:14.267231 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:06:14.286490 13216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.273023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49052 (local address 127.8.62.190:40825)
0807 18:06:14.273198 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.273203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.273677 (+ 474us) server_negotiation.cc:407] Connection header received
0807 18:06:14.273789 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.273793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.273862 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.273958 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:14.275481 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.276230 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.277782 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.278225 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.278383 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.280915 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:14.280920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:14.280924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:14.280962 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:14.283617 (+ 2655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.284311 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.284318 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.284321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.284452 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.284902 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.284911 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.284913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.285275 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:14.285578 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.285920 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.286169 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:06:14.291117 12749 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:14.298262 12749 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:14.322132 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.311853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.8.62.129:41145)
0807 18:06:14.313226 (+ 1373us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.313235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.313267 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:06:14.313370 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.313378 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.313693 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.313943 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:14.315248 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.316974 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.319300 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.320717 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.320860 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.321104 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.321573 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.321770 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1135,"thread_start_us":968,"threads_started":1}
I20260807 18:06:14.322221 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.311660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41145 (local address 127.0.0.1:48872)
0807 18:06:14.312413 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.312459 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.312658 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.314029 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.314039 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.314091 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:14.314977 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.314998 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.317107 (+ 2109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.317115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.319136 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.319148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.320926 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.320933 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.321161 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.321372 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.321517 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.321716 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":3,"thread_start_us":374,"threads_started":1}
I20260807 18:06:14.323684 13229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.312717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:39833 (local address 127.0.0.1:37222)
0807 18:06:14.313457 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.313475 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.313590 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.315455 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.315458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.315477 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:14.315927 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.315941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.317733 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.317741 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.319956 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.319971 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.321947 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.321954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.322182 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.322409 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.322483 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.322630 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":1,"thread_start_us":462,"threads_started":1}
I20260807 18:06:14.323745 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.313312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37222 (local address 127.8.62.130:39833)
0807 18:06:14.314651 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.314662 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.314691 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:06:14.314795 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.314804 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.315034 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.315250 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:14.316070 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.317582 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.320145 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.321654 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.322922 (+ 1268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.323145 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.323243 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.323419 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":1046,"threads_started":1}
I20260807 18:06:14.326656 13231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.314142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:42745 (local address 127.0.0.1:57528)
0807 18:06:14.315038 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.315060 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.315185 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.317171 (+ 1986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.317175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.317203 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:14.317658 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.317670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.320331 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.320338 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.322697 (+ 2359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.322712 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.325810 (+ 3098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.325817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.325993 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.326025 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.326126 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.326266 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":23,"thread_start_us":773,"threads_started":1}
I20260807 18:06:14.326673 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.314759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.8.62.131:42745)
0807 18:06:14.316627 (+ 1868us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.316635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.316666 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:06:14.316753 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.316760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.316961 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.317142 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:14.317853 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.320193 (+ 2340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.322874 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.325590 (+ 2716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.325733 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.325970 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.326194 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.326357 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1650,"thread_start_us":1501,"threads_started":1}
I20260807 18:06:14.327124 12876 tablet_service.cc:1467] Processing CreateTablet for tablet 484fe53255ce4545a357792547d30495 (DEFAULT_TABLE table=test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:14.327504 13011 tablet_service.cc:1467] Processing CreateTablet for tablet 484fe53255ce4545a357792547d30495 (DEFAULT_TABLE table=test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:14.328660 12876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484fe53255ce4545a357792547d30495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:14.328842 13011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484fe53255ce4545a357792547d30495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:14.330756 13147 tablet_service.cc:1467] Processing CreateTablet for tablet 484fe53255ce4545a357792547d30495 (DEFAULT_TABLE table=test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:14.332178 13147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 484fe53255ce4545a357792547d30495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:14.341097 13233 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Bootstrap starting.
I20260807 18:06:14.344811 13233 tablet_bootstrap.cc:654] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:14.345377 13234 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Bootstrap starting.
I20260807 18:06:14.346580 13233 log.cc:824] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:14.348798 13235 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Bootstrap starting.
I20260807 18:06:14.349538 13234 tablet_bootstrap.cc:654] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:14.350629 13233 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: No bootstrap required, opened a new log
I20260807 18:06:14.350881 13234 log.cc:824] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:14.350939 13233 ts_tablet_manager.cc:1397] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260807 18:06:14.351910 13235 tablet_bootstrap.cc:654] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:14.353111 13235 log.cc:824] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:14.354306 13234 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: No bootstrap required, opened a new log
I20260807 18:06:14.354597 13234 ts_tablet_manager.cc:1397] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 18:06:14.355734 13235 tablet_bootstrap.cc:492] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: No bootstrap required, opened a new log
I20260807 18:06:14.356062 13235 ts_tablet_manager.cc:1397] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 18:06:14.362175 13233 raft_consensus.cc:348] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.362730 13233 raft_consensus.cc:374] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:14.362838 13233 raft_consensus.cc:729] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e5005cd00940a1a6c528f121b86913, State: Initialized, Role: FOLLOWER
I20260807 18:06:14.363675 13233 consensus_queue.cc:260] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.364399 13235 raft_consensus.cc:348] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.364930 13235 raft_consensus.cc:374] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:14.365020 13077 heartbeater.cc:503] Master 127.8.62.190:40825 was elected leader, sending a full tablet report...
I20260807 18:06:14.365068 13235 raft_consensus.cc:729] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff31a8dcd0ba470380c716720be7c82f, State: Initialized, Role: FOLLOWER
I20260807 18:06:14.365746 13233 ts_tablet_manager.cc:1428] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260807 18:06:14.365710 13235 consensus_queue.cc:260] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.366882 13214 heartbeater.cc:503] Master 127.8.62.190:40825 was elected leader, sending a full tablet report...
I20260807 18:06:14.367260 13234 raft_consensus.cc:348] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.367782 13234 raft_consensus.cc:374] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:14.367888 13234 raft_consensus.cc:729] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa97292aec3c4c9bbde724cb0e5e7cf9, State: Initialized, Role: FOLLOWER
I20260807 18:06:14.368695 13234 consensus_queue.cc:260] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.369906 13235 ts_tablet_manager.cc:1428] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20260807 18:06:14.370003 13239 raft_consensus.cc:482] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:14.370285 13239 raft_consensus.cc:504] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.372422 13239 leader_election.cc:288] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa97292aec3c4c9bbde724cb0e5e7cf9 (127.8.62.129:41145), ff31a8dcd0ba470380c716720be7c82f (127.8.62.131:42745)
I20260807 18:06:14.377075 13234 ts_tablet_manager.cc:1428] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Time spent starting tablet: real 0.022s user 0.015s sys 0.007s
I20260807 18:06:14.385783 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.373262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:37563 (local address 127.8.62.129:41145)
0807 18:06:14.373571 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.373580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.378230 (+ 4650us) server_negotiation.cc:407] Connection header received
0807 18:06:14.378533 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.378543 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.378648 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.378767 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:14.379666 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.380821 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.383527 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.384618 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.384798 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.384856 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.385323 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.385493 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:06:14.385815 13243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.372619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41145 (local address 127.8.62.130:37563)
0807 18:06:14.378083 (+ 5464us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.378107 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.378322 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.378956 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.378962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.379004 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:14.379459 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.379473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.381029 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.381037 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.383337 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.383350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.384901 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.384909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.385122 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.385161 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.385285 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.385451 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5269,"thread_start_us":6000,"threads_started":1}
I20260807 18:06:14.386329 13242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.373115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:42745 (local address 127.8.62.130:49885)
0807 18:06:14.378912 (+ 5797us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:14.378940 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:14.379165 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:14.379703 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:14.379708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:14.379734 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:14.380196 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.380209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.381715 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.381720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.383897 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:14.383913 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.385548 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.385555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:14.385753 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.385796 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:06:14.385912 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:06:14.386057 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5667,"thread_start_us":4553,"threads_started":1}
I20260807 18:06:14.386705 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.373744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:49885 (local address 127.8.62.131:42745)
0807 18:06:14.373998 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.374007 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.379036 (+ 5029us) server_negotiation.cc:407] Connection header received
0807 18:06:14.379347 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.379353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.379431 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.379493 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:14.380398 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.381503 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.384101 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.385312 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.385473 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.385554 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.386294 (+ 740us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.386444 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:06:14.387063 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484fe53255ce4545a357792547d30495" candidate_uuid: "34e5005cd00940a1a6c528f121b86913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" is_pre_election: true
I20260807 18:06:14.387643 12896 raft_consensus.cc:2454] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e5005cd00940a1a6c528f121b86913 in term 0.
I20260807 18:06:14.387658 13167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484fe53255ce4545a357792547d30495" candidate_uuid: "34e5005cd00940a1a6c528f121b86913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff31a8dcd0ba470380c716720be7c82f" is_pre_election: true
I20260807 18:06:14.388234 13167 raft_consensus.cc:2454] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e5005cd00940a1a6c528f121b86913 in term 0.
I20260807 18:06:14.388458 12963 leader_election.cc:302] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e5005cd00940a1a6c528f121b86913, fa97292aec3c4c9bbde724cb0e5e7cf9; no voters:
I20260807 18:06:14.388971 13239 raft_consensus.cc:2781] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:14.389140 13239 raft_consensus.cc:482] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:14.389251 13239 raft_consensus.cc:3037] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:14.391583 13239 raft_consensus.cc:504] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.392287 13239 leader_election.cc:288] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [CANDIDATE]: Term 1 election: Requested vote from peers fa97292aec3c4c9bbde724cb0e5e7cf9 (127.8.62.129:41145), ff31a8dcd0ba470380c716720be7c82f (127.8.62.131:42745)
I20260807 18:06:14.392882 12896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484fe53255ce4545a357792547d30495" candidate_uuid: "34e5005cd00940a1a6c528f121b86913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9"
I20260807 18:06:14.393064 13167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "484fe53255ce4545a357792547d30495" candidate_uuid: "34e5005cd00940a1a6c528f121b86913" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff31a8dcd0ba470380c716720be7c82f"
I20260807 18:06:14.393139 12896 raft_consensus.cc:3037] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:14.393296 13167 raft_consensus.cc:3037] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:14.396770 12896 raft_consensus.cc:2454] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e5005cd00940a1a6c528f121b86913 in term 1.
I20260807 18:06:14.396770 13167 raft_consensus.cc:2454] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e5005cd00940a1a6c528f121b86913 in term 1.
I20260807 18:06:14.397329 12964 leader_election.cc:302] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e5005cd00940a1a6c528f121b86913, ff31a8dcd0ba470380c716720be7c82f; no voters:
I20260807 18:06:14.397699 13239 raft_consensus.cc:2781] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:14.398270 13239 raft_consensus.cc:686] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 1 LEADER]: Becoming Leader. State: Replica: 34e5005cd00940a1a6c528f121b86913, State: Running, Role: LEADER
I20260807 18:06:14.398921 13239 consensus_queue.cc:237] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } }
I20260807 18:06:14.404233 12748 catalog_manager.cc:5495] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e5005cd00940a1a6c528f121b86913 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "34e5005cd00940a1a6c528f121b86913" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e5005cd00940a1a6c528f121b86913" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 } health_report { overall_health: UNKNOWN } } }
I20260807 18:06:14.432509 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.426989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37228 (local address 127.8.62.130:39833)
0807 18:06:14.427282 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.427287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.427308 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:14.427490 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.427494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.427625 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.427766 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:14.428157 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.428982 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.430332 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.430749 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.430852 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.431309 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:14.431855 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:14.431987 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.432105 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.432182 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
W20260807 18:06:14.435491 12942 tablet.cc:2369] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:14.441638 12896 raft_consensus.cc:1261] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 1 FOLLOWER]: Refusing update from remote peer 34e5005cd00940a1a6c528f121b86913: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:14.441629 13167 raft_consensus.cc:1261] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 1 FOLLOWER]: Refusing update from remote peer 34e5005cd00940a1a6c528f121b86913: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:14.442677 13239 consensus_queue.cc:1035] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff31a8dcd0ba470380c716720be7c82f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 42745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:14.443038 13244 consensus_queue.cc:1035] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa97292aec3c4c9bbde724cb0e5e7cf9" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:14.455593 13247 mvcc.cc:205] Tried to move back new op lower bound from 7315971991301120000 to 7315971991142973440. Current Snapshot: MvccSnapshot[applied={T|T < 7315971991301120000}]
I20260807 18:06:14.457449 13248 mvcc.cc:205] Tried to move back new op lower bound from 7315971991301120000 to 7315971991142973440. Current Snapshot: MvccSnapshot[applied={T|T < 7315971991301120000}]
I20260807 18:06:14.457813 13249 mvcc.cc:205] Tried to move back new op lower bound from 7315971991301120000 to 7315971991142973440. Current Snapshot: MvccSnapshot[applied={T|T < 7315971991301120000}]
I20260807 18:06:14.472021 12747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:14.472386 12747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:14.476794 12747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b1c8e3594634deba4f1044cae8c4bc1: Sending DeleteTablet for 3 replicas of tablet 484fe53255ce4545a357792547d30495
I20260807 18:06:14.478255 13147 tablet_service.cc:1514] Processing DeleteTablet for tablet 484fe53255ce4545a357792547d30495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:14 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:57528
I20260807 18:06:14.478253 12876 tablet_service.cc:1514] Processing DeleteTablet for tablet 484fe53255ce4545a357792547d30495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:14 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:48872
I20260807 18:06:14.479066 13011 tablet_service.cc:1514] Processing DeleteTablet for tablet 484fe53255ce4545a357792547d30495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:14 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:37222
I20260807 18:06:14.479329 13257 tablet_replica.cc:330] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: stopping tablet replica
I20260807 18:06:14.479828 13257 raft_consensus.cc:2229] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:14.480443 13257 raft_consensus.cc:2258] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:14.481199 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12813
I20260807 18:06:14.484668 13257 ts_tablet_manager.cc:1905] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:14.494812 13258 tablet_replica.cc:330] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: stopping tablet replica
I20260807 18:06:14.495488 13258 raft_consensus.cc:2229] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:14.496197 13258 raft_consensus.cc:2258] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:14.497218 13259 tablet_replica.cc:330] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: stopping tablet replica
I20260807 18:06:14.497625 13259 raft_consensus.cc:2229] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:14.498225 13259 raft_consensus.cc:2258] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:14.501283 13258 ts_tablet_manager.cc:1905] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:14.507323 13258 ts_tablet_manager.cc:1918] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:14.507547 13258 log.cc:1196] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/484fe53255ce4545a357792547d30495
I20260807 18:06:14.507509 13257 ts_tablet_manager.cc:1918] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:14.507773 13257 log.cc:1196] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/484fe53255ce4545a357792547d30495
I20260807 18:06:14.508224 13258 ts_tablet_manager.cc:1939] T 484fe53255ce4545a357792547d30495 P fa97292aec3c4c9bbde724cb0e5e7cf9: Deleting consensus metadata
I20260807 18:06:14.508430 13257 ts_tablet_manager.cc:1939] T 484fe53255ce4545a357792547d30495 P ff31a8dcd0ba470380c716720be7c82f: Deleting consensus metadata
I20260807 18:06:14.508880 13259 ts_tablet_manager.cc:1905] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 18:06:14 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:14.510582 12734 catalog_manager.cc:4841] TS fa97292aec3c4c9bbde724cb0e5e7cf9 (127.8.62.129:41145): tablet 484fe53255ce4545a357792547d30495 (table test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]) successfully deleted
I20260807 18:06:14.511021 12735 catalog_manager.cc:4841] TS ff31a8dcd0ba470380c716720be7c82f (127.8.62.131:42745): tablet 484fe53255ce4545a357792547d30495 (table test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]) successfully deleted
I20260807 18:06:14.515463 13259 ts_tablet_manager.cc:1918] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:14.515674 13259 log.cc:1196] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/484fe53255ce4545a357792547d30495
I20260807 18:06:14.516312 13259 ts_tablet_manager.cc:1939] T 484fe53255ce4545a357792547d30495 P 34e5005cd00940a1a6c528f121b86913: Deleting consensus metadata
I20260807 18:06:14.518179 12733 catalog_manager.cc:4841] TS 34e5005cd00940a1a6c528f121b86913 (127.8.62.130:39833): tablet 484fe53255ce4545a357792547d30495 (table test-table [id=485bf7f5bbeb4922bd7eaaa056d6b4fd]) successfully deleted
I20260807 18:06:14.519333 13228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.482565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.8.62.129:41145)
0807 18:06:14.482822 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.482831 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.499084 (+ 16253us) server_negotiation.cc:407] Connection header received
0807 18:06:14.499212 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.499218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.499337 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.499475 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:14.501582 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.502820 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.504838 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.505549 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.505659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.511419 (+ 5760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:14.511435 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:14.511454 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:14.511507 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:14.515542 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.516616 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.516627 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.516631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.516820 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.517305 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.517311 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.517315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.517809 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:14.518143 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.518678 (+ 535us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.518925 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:06:14.520061 12936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:14.626720 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12813
I20260807 18:06:14.645860 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12947
Aug 07 18:06:14 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:06:14.664736 13230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.646788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37244 (local address 127.8.62.130:39833)
0807 18:06:14.646974 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.646981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.647064 (+ 83us) server_negotiation.cc:407] Connection header received
0807 18:06:14.647278 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.647282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.647353 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.647492 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:14.648803 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.649649 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.650937 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.651425 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.651578 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.657687 (+ 6109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:14.657714 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:14.657736 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:14.657807 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:14.661392 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.662252 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.662262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.662267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.662434 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.662875 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.662880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.662884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.663312 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:14.663669 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.663988 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.664240 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:06:14.665544 13072 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:14.783937 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12947
I20260807 18:06:14.807385 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13082
Aug 07 18:06:14 dist-test-slave-rnrw krb5kdc[12705](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:06:14.823731 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:14.808186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57532 (local address 127.8.62.131:42745)
0807 18:06:14.808379 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:14.808386 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:14.808456 (+ 70us) server_negotiation.cc:407] Connection header received
0807 18:06:14.808790 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:14.808795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:14.808870 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:14.808979 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:14.810360 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.811098 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.812367 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:14.812843 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:14.812986 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:14.817639 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:14.817655 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:14.817672 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:14.817716 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:14.820671 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.821410 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.821419 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.821422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.821597 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:14.822014 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:14.822019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:14.822022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:14.822480 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:14.822785 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:14.823136 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 18:06:14.823386 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:06:14.824536 13209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:14.917904 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13082
I20260807 18:06:14.936525 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 12718
I20260807 18:06:14.937779 12781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:15.049990 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 12718
2026-08-07T18:06:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4305 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T18:06:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:15Z Disabled control of system clock
I20260807 18:06:15.125831 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43441
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:43441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:06:15.476742 13269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:15.477061 13269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:15.477147 13269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:15.484490 13269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:15.484658 13269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:15.484697 13269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:15.484754 13269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:15.484799 13269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:15.494693 13269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:43441
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:15.496009 13269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:15.497956 13269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:15.510903 13274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:15.510944 13277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:15.510871 13275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:15.511905 13269 server_base.cc:1034] running on GCE node
I20260807 18:06:15.512898 13269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:15.514613 13269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:15.515874 13269 hybrid_clock.cc:648] HybridClock initialized: now 1786125975515815 us; error 54 us; skew 500 ppm
I20260807 18:06:15.516304 13269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:15.524112 13269 webserver.cc:460] Webserver started at http://127.8.62.190:43027/ using document root <none> and password file <none>
I20260807 18:06:15.525147 13269 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:15.525280 13269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:15.525717 13269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:15.529034 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "de1d74e4075d49a9ac2682df4d3e7755"
format_stamp: "Formatted at 2026-08-07 18:06:15 on dist-test-slave-rnrw"
server_key: "a489495f589ac3c3c9847581b2b24456"
server_key_iv: "935b0fbd79ca538ef406701542bcc957"
server_key_version: "encryptionkey@0"
I20260807 18:06:15.529901 13269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "de1d74e4075d49a9ac2682df4d3e7755"
format_stamp: "Formatted at 2026-08-07 18:06:15 on dist-test-slave-rnrw"
server_key: "a489495f589ac3c3c9847581b2b24456"
server_key_iv: "935b0fbd79ca538ef406701542bcc957"
server_key_version: "encryptionkey@0"
I20260807 18:06:15.535568 13269 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:15.539565 13283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:15.541146 13269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:15.541352 13269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "de1d74e4075d49a9ac2682df4d3e7755"
format_stamp: "Formatted at 2026-08-07 18:06:15 on dist-test-slave-rnrw"
server_key: "a489495f589ac3c3c9847581b2b24456"
server_key_iv: "935b0fbd79ca538ef406701542bcc957"
server_key_version: "encryptionkey@0"
I20260807 18:06:15.541646 13269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:15.547686 13269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:15.548590 13269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:15.549149 13269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:15.569108 13269 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:43441
I20260807 18:06:15.569125 13336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:43441 every 8 connection(s)
I20260807 18:06:15.570672 13269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:15.574849 13337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:15.578490 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13269
I20260807 18:06:15.578723 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:15.579391 8442 external_mini_cluster.cc:1428] Setting key 8ea3637572b0e9e9e3ae5fab98986e7c
I20260807 18:06:15.587329 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: Bootstrap starting.
I20260807 18:06:15.589511 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:15.581873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.8.62.190:43441)
0807 18:06:15.583050 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:15.583063 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:15.583113 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:06:15.585562 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:15.585589 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:15.586085 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:15.586453 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:15.587735 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:15.587742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:15.587755 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:15.587771 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:15.587842 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:15.587860 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:15.588056 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:15.588161 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:06:15.588447 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":434,"threads_started":1}
I20260807 18:06:15.591270 13337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:15.592447 13337 log.cc:824] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:15.595374 13337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: No bootstrap required, opened a new log
I20260807 18:06:15.603498 13337 raft_consensus.cc:348] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } }
I20260807 18:06:15.603902 13337 raft_consensus.cc:374] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:15.604025 13337 raft_consensus.cc:729] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de1d74e4075d49a9ac2682df4d3e7755, State: Initialized, Role: FOLLOWER
I20260807 18:06:15.604768 13337 consensus_queue.cc:260] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } }
I20260807 18:06:15.605016 13337 raft_consensus.cc:388] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:15.605159 13337 raft_consensus.cc:482] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:15.605325 13337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:15.607618 13337 raft_consensus.cc:504] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } }
I20260807 18:06:15.608175 13337 leader_election.cc:302] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de1d74e4075d49a9ac2682df4d3e7755; no voters:
I20260807 18:06:15.608888 13337 leader_election.cc:288] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:15.609087 13342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:15.609879 13342 raft_consensus.cc:686] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [term 1 LEADER]: Becoming Leader. State: Replica: de1d74e4075d49a9ac2682df4d3e7755, State: Running, Role: LEADER
I20260807 18:06:15.610528 13342 consensus_queue.cc:237] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } }
I20260807 18:06:15.611356 13337 sys_catalog.cc:564] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:15.613819 13343 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de1d74e4075d49a9ac2682df4d3e7755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } } }
I20260807 18:06:15.614267 13343 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:15.615293 13344 sys_catalog.cc:455] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de1d74e4075d49a9ac2682df4d3e7755. Latest consensus state: current_term: 1 leader_uuid: "de1d74e4075d49a9ac2682df4d3e7755" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de1d74e4075d49a9ac2682df4d3e7755" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43441 } } }
I20260807 18:06:15.615638 13344 sys_catalog.cc:458] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:15.615931 13351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:15.621673 13351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:15.632164 13351 catalog_manager.cc:1347] Generated new cluster ID: 2fbf3d484d934d69b4af5c28d1f9c1b0
I20260807 18:06:15.632300 13351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:15.707075 13351 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:15.707597 13351 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:15.718925 13351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: Generated new TSK 0
I20260807 18:06:15.720081 13351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:06:15.741557 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--builtin_ntp_servers=127.8.62.148:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 18:06:16.092026 13361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:16.092403 13361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:16.092638 13361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:16.100749 13361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:16.100915 13361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:16.101114 13361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:16.111182 13361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:16.112744 13361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:16.114977 13361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:06:16.130558 13361 server_base.cc:1034] running on GCE node
W20260807 18:06:16.130467 13367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:16.130517 13366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:16.130578 13369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:16.131714 13361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:16.132827 13361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:16.134126 13361 hybrid_clock.cc:648] HybridClock initialized: now 1786125976134037 us; error 80 us; skew 500 ppm
I20260807 18:06:16.134519 13361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:16.137264 13361 webserver.cc:460] Webserver started at http://127.8.62.129:38171/ using document root <none> and password file <none>
I20260807 18:06:16.138048 13361 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:16.138195 13361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:16.138545 13361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:16.141875 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "ef79536310c54a9e8d9edebbfa0d5b30"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e1a252b4c2d63105d4a6c3d14bdfc15"
server_key_iv: "f4df9d401782bddc408e75f5aa4601c1"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.142813 13361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "ef79536310c54a9e8d9edebbfa0d5b30"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e1a252b4c2d63105d4a6c3d14bdfc15"
server_key_iv: "f4df9d401782bddc408e75f5aa4601c1"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.149016 13361 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:06:16.153118 13375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.154803 13361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:06:16.155032 13361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "ef79536310c54a9e8d9edebbfa0d5b30"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e1a252b4c2d63105d4a6c3d14bdfc15"
server_key_iv: "f4df9d401782bddc408e75f5aa4601c1"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.155319 13361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:16.162062 13361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:16.162997 13361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:16.163300 13361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:16.164323 13361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:16.165958 13361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:16.166092 13361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.166236 13361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:16.166327 13361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.202297 13361 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:38045
I20260807 18:06:16.202349 13489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:38045 every 8 connection(s)
I20260807 18:06:16.203774 13361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:16.205255 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13361
I20260807 18:06:16.205472 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:16.206332 8442 external_mini_cluster.cc:1428] Setting key 44300f016607493a776046173e97d63f
I20260807 18:06:16.209784 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--builtin_ntp_servers=127.8.62.148:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:06:16.214176 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:16.207183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:41687 (local address 127.8.62.190:43441)
0807 18:06:16.207774 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:16.207780 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:16.209415 (+ 1635us) server_negotiation.cc:407] Connection header received
0807 18:06:16.211474 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:16.211480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:16.211558 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:16.211698 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:16.213277 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:16.213282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:16.213287 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:16.213311 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:16.213374 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:16.213386 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:16.213537 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:16.213744 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 18:06:16.213827 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":306,"threads_started":1}
I20260807 18:06:16.215431 13493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:16.208010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43441 (local address 127.8.62.129:41687)
0807 18:06:16.209243 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:16.209292 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:16.211072 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:16.211989 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:16.211999 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:16.212110 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:16.212936 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:16.212938 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:16.212983 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:16.213085 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:16.213492 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:16.213504 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:16.213513 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:06:16.213842 (+ 329us) client_negotiation.cc:240] Negotiation successful
0807 18:06:16.214153 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":428,"threads_started":1}
I20260807 18:06:16.217446 13490 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43441
I20260807 18:06:16.217819 13490 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:16.218581 13490 heartbeater.cc:511] Master 127.8.62.190:43441 requested a full tablet report, sending...
I20260807 18:06:16.221500 13300 ts_manager.cc:194] Registered new tserver with Master: ef79536310c54a9e8d9edebbfa0d5b30 (127.8.62.129:38045)
W20260807 18:06:16.566318 13494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:16.566725 13494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:16.566895 13494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:16.575107 13494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:16.575232 13494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:16.575426 13494 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:16.585804 13494 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:16.587181 13494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:16.589110 13494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:16.602946 13500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:16.602905 13499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:16.603008 13502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:16.604935 13494 server_base.cc:1034] running on GCE node
I20260807 18:06:16.605652 13494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:16.606738 13494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:16.608085 13494 hybrid_clock.cc:648] HybridClock initialized: now 1786125976607993 us; error 60 us; skew 500 ppm
I20260807 18:06:16.608444 13494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:16.610852 13494 webserver.cc:460] Webserver started at http://127.8.62.130:46399/ using document root <none> and password file <none>
I20260807 18:06:16.611668 13494 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:16.611804 13494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:16.612138 13494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:16.615346 13494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "65e3eabcc0ca432fb30659d0cd11bcac"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e36e4a6608bf9af79ac32ba65936157"
server_key_iv: "52f2cf5e7dfbac2eb2b01b5e83c23c2b"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.616257 13494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "65e3eabcc0ca432fb30659d0cd11bcac"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e36e4a6608bf9af79ac32ba65936157"
server_key_iv: "52f2cf5e7dfbac2eb2b01b5e83c23c2b"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.621932 13494 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:06:16.625898 13508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.627611 13494 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:16.627770 13494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "65e3eabcc0ca432fb30659d0cd11bcac"
format_stamp: "Formatted at 2026-08-07 18:06:16 on dist-test-slave-rnrw"
server_key: "6e36e4a6608bf9af79ac32ba65936157"
server_key_iv: "52f2cf5e7dfbac2eb2b01b5e83c23c2b"
server_key_version: "encryptionkey@0"
I20260807 18:06:16.627985 13494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:16.633991 13494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:16.634809 13494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:16.635057 13494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:16.635957 13494 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:16.637601 13494 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:16.637692 13494 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.637778 13494 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:16.637832 13494 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:16.674328 13494 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:35459
I20260807 18:06:16.674386 13620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:35459 every 8 connection(s)
I20260807 18:06:16.675827 13494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:16.684156 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:16.678635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:58783 (local address 127.8.62.190:43441)
0807 18:06:16.678830 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:16.678834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:16.680271 (+ 1437us) server_negotiation.cc:407] Connection header received
0807 18:06:16.682060 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:16.682065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:16.682124 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:16.682220 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:16.683509 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:16.683513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:16.683517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:16.683534 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:16.683568 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:16.683579 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:16.683708 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:16.683874 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:06:16.683931 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:06:16.685222 13623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:16.679138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43441 (local address 127.8.62.130:58783)
0807 18:06:16.680088 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:16.680139 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:16.681787 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:16.682436 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:16.682445 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:16.682583 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:16.683212 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:16.683214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:16.683252 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:16.683343 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:16.683695 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:16.683703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:16.683708 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:06:16.683927 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 18:06:16.684133 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":287,"threads_started":1}
I20260807 18:06:16.685678 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13494
I20260807 18:06:16.685868 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:16.686324 8442 external_mini_cluster.cc:1428] Setting key 441cce8c4aa1d385538618904fb94b7d
I20260807 18:06:16.686950 13621 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43441
I20260807 18:06:16.687309 13621 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:16.688130 13621 heartbeater.cc:511] Master 127.8.62.190:43441 requested a full tablet report, sending...
I20260807 18:06:16.689648 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--builtin_ntp_servers=127.8.62.148:33301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 18:06:16.689826 13300 ts_manager.cc:194] Registered new tserver with Master: 65e3eabcc0ca432fb30659d0cd11bcac (127.8.62.130:35459)
W20260807 18:06:17.040992 13624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:17.041307 13624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:17.041471 13624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:17.049122 13624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:17.049247 13624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:17.049433 13624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:17.059178 13624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:17.060734 13624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:17.062804 13624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:17.077883 13633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:17.077854 13630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:17.077955 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:17.082798 13624 server_base.cc:1034] running on GCE node
I20260807 18:06:17.083657 13624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:17.084885 13624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:17.086127 13624 hybrid_clock.cc:648] HybridClock initialized: now 1786125977086069 us; error 49 us; skew 500 ppm
I20260807 18:06:17.086506 13624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:17.089529 13624 webserver.cc:460] Webserver started at http://127.8.62.131:44635/ using document root <none> and password file <none>
I20260807 18:06:17.090371 13624 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:17.090479 13624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:17.090876 13624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:17.094386 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "30f55524dbe542db967e61733460bfae"
format_stamp: "Formatted at 2026-08-07 18:06:17 on dist-test-slave-rnrw"
server_key: "498865cf4d11fdac10ad6d72d4bcceda"
server_key_iv: "958b21320bb5fba4895c86bf30d764a8"
server_key_version: "encryptionkey@0"
I20260807 18:06:17.095459 13624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "30f55524dbe542db967e61733460bfae"
format_stamp: "Formatted at 2026-08-07 18:06:17 on dist-test-slave-rnrw"
server_key: "498865cf4d11fdac10ad6d72d4bcceda"
server_key_iv: "958b21320bb5fba4895c86bf30d764a8"
server_key_version: "encryptionkey@0"
I20260807 18:06:17.101842 13624 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:06:17.106043 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:17.107864 13624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:17.108088 13624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "30f55524dbe542db967e61733460bfae"
format_stamp: "Formatted at 2026-08-07 18:06:17 on dist-test-slave-rnrw"
server_key: "498865cf4d11fdac10ad6d72d4bcceda"
server_key_iv: "958b21320bb5fba4895c86bf30d764a8"
server_key_version: "encryptionkey@0"
I20260807 18:06:17.108383 13624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:17.115391 13624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:17.116356 13624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:17.116762 13624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:17.117849 13624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:17.119442 13624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:17.119568 13624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:17.119715 13624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:17.119808 13624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:17.155452 13624 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:45707
I20260807 18:06:17.155509 13752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:45707 every 8 connection(s)
I20260807 18:06:17.157029 13624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:17.162339 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13624
I20260807 18:06:17.162528 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:17.162981 8442 external_mini_cluster.cc:1428] Setting key 63a24fe5673bd7863a874758fe96e4f0
I20260807 18:06:17.167272 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.160164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41121 (local address 127.8.62.190:43441)
0807 18:06:17.160388 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.160396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.162153 (+ 1757us) server_negotiation.cc:407] Connection header received
0807 18:06:17.164484 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.164490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.164597 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.164748 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.166465 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.166471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.166476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.166497 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.166541 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.166553 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.166679 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.166893 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.166967 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:06:17.168336 13755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.160882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43441 (local address 127.8.62.131:41121)
0807 18:06:17.161946 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.161998 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.164049 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.165180 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.165193 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.165342 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.166151 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.166153 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.166196 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.166289 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.166659 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.166670 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.166678 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.166991 (+ 313us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.167293 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":334,"threads_started":1}
I20260807 18:06:17.169893 13753 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43441
I20260807 18:06:17.170275 13753 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:17.170958 13753 heartbeater.cc:511] Master 127.8.62.190:43441 requested a full tablet report, sending...
I20260807 18:06:17.172597 13300 ts_manager.cc:194] Registered new tserver with Master: 30f55524dbe542db967e61733460bfae (127.8.62.131:45707)
I20260807 18:06:17.179332 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:06:17.187453 13492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.184259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58574 (local address 127.8.62.190:43441)
0807 18:06:17.184620 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.184625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.185029 (+ 404us) server_negotiation.cc:407] Connection header received
0807 18:06:17.185162 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.185166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.185230 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.185325 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.186633 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.186641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.186646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.186677 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.186724 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.186741 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.186884 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.187076 (+ 192us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.187131 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:06:17.192726 13300 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58574:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:17.199898 13300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:17.218708 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.214282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47644)
0807 18:06:17.215128 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.215170 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.215293 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.215935 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.215942 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.216037 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.216982 (+ 945us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.216991 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.217022 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.217116 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.217804 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.217817 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.217824 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.218068 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.218191 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":380,"threads_started":1}
I20260807 18:06:17.218755 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.214477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47644 (local address 127.8.62.129:38045)
0807 18:06:17.215136 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.215146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.215271 (+ 125us) server_negotiation.cc:407] Connection header received
0807 18:06:17.215421 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.215431 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.215714 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.215887 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.217358 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.217370 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.217390 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.217416 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.217489 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.217513 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.217684 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.218101 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.218251 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":292,"threads_started":1}
I20260807 18:06:17.225674 13490 heartbeater.cc:503] Master 127.8.62.190:43441 was elected leader, sending a full tablet report...
I20260807 18:06:17.226792 13423 tablet_service.cc:1467] Processing CreateTablet for tablet 74410ef98f704446ba1f2ea858bbb149 (DEFAULT_TABLE table=test-table [id=43e5906f8c874ff1b0b5c7bcbe2f8b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:17.227826 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.217342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35459 (local address 127.0.0.1:54278)
0807 18:06:17.219391 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.219414 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.219514 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.222461 (+ 2947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.222469 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.222533 (+ 64us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.223278 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.223285 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.223311 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.223387 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.227154 (+ 3767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.227159 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.227165 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.227290 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.227356 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1904,"threads_started":1}
I20260807 18:06:17.228611 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.217604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.8.62.130:35459)
0807 18:06:17.221708 (+ 4104us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.221720 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.221764 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:06:17.221856 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.221868 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.222201 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.222976 (+ 775us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.224774 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.224787 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.224808 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.224842 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.224915 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.224941 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.227193 (+ 2252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.228093 (+ 900us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.228230 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3892,"thread_start_us":3758,"threads_started":1}
I20260807 18:06:17.228777 13423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74410ef98f704446ba1f2ea858bbb149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:17.229472 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.216527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:45707 (local address 127.0.0.1:34918)
0807 18:06:17.217661 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.217687 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.217811 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.227286 (+ 9475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.227291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.227319 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.228229 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.228233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.228251 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.228302 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.228955 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.228961 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.228967 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.229107 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.229157 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":1206,"threads_started":1}
I20260807 18:06:17.230350 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.217032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.8.62.131:45707)
0807 18:06:17.225604 (+ 8572us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.225623 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.225666 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:06:17.225739 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.225747 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.225973 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.226146 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.228506 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.228516 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.228534 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.228596 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.228677 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.228807 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.229934 (+ 1127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.229989 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.230074 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8403,"thread_start_us":8222,"threads_started":1}
I20260807 18:06:17.234310 13688 tablet_service.cc:1467] Processing CreateTablet for tablet 74410ef98f704446ba1f2ea858bbb149 (DEFAULT_TABLE table=test-table [id=43e5906f8c874ff1b0b5c7bcbe2f8b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:17.234572 13556 tablet_service.cc:1467] Processing CreateTablet for tablet 74410ef98f704446ba1f2ea858bbb149 (DEFAULT_TABLE table=test-table [id=43e5906f8c874ff1b0b5c7bcbe2f8b78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:17.236145 13688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74410ef98f704446ba1f2ea858bbb149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:17.236140 13556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74410ef98f704446ba1f2ea858bbb149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:17.247094 13771 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: Bootstrap starting.
I20260807 18:06:17.251490 13771 tablet_bootstrap.cc:654] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:17.251466 13772 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Bootstrap starting.
I20260807 18:06:17.253196 13771 log.cc:824] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:17.257350 13772 tablet_bootstrap.cc:654] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:17.257643 13773 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Bootstrap starting.
I20260807 18:06:17.259111 13772 log.cc:824] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:17.262136 13771 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: No bootstrap required, opened a new log
I20260807 18:06:17.262364 13773 tablet_bootstrap.cc:654] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:17.262521 13771 ts_tablet_manager.cc:1397] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20260807 18:06:17.262818 13772 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: No bootstrap required, opened a new log
I20260807 18:06:17.263101 13772 ts_tablet_manager.cc:1397] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 18:06:17.263765 13773 log.cc:824] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:17.267292 13773 tablet_bootstrap.cc:492] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: No bootstrap required, opened a new log
I20260807 18:06:17.267627 13773 ts_tablet_manager.cc:1397] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 18:06:17.271458 13772 raft_consensus.cc:348] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.272023 13772 raft_consensus.cc:374] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:17.272161 13772 raft_consensus.cc:729] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f55524dbe542db967e61733460bfae, State: Initialized, Role: FOLLOWER
I20260807 18:06:17.272918 13772 consensus_queue.cc:260] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.273417 13771 raft_consensus.cc:348] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.274137 13753 heartbeater.cc:503] Master 127.8.62.190:43441 was elected leader, sending a full tablet report...
I20260807 18:06:17.274711 13772 ts_tablet_manager.cc:1428] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260807 18:06:17.274957 13771 raft_consensus.cc:374] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:17.275131 13771 raft_consensus.cc:729] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef79536310c54a9e8d9edebbfa0d5b30, State: Initialized, Role: FOLLOWER
I20260807 18:06:17.275848 13771 consensus_queue.cc:260] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.278473 13773 raft_consensus.cc:348] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.279194 13773 raft_consensus.cc:374] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:17.279350 13773 raft_consensus.cc:729] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e3eabcc0ca432fb30659d0cd11bcac, State: Initialized, Role: FOLLOWER
I20260807 18:06:17.280330 13771 ts_tablet_manager.cc:1428] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30: Time spent starting tablet: real 0.018s user 0.010s sys 0.008s
I20260807 18:06:17.280122 13773 consensus_queue.cc:260] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.281500 13621 heartbeater.cc:503] Master 127.8.62.190:43441 was elected leader, sending a full tablet report...
I20260807 18:06:17.282315 13773 ts_tablet_manager.cc:1428] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 18:06:17.331805 13779 raft_consensus.cc:482] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:17.332232 13779 raft_consensus.cc:504] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.334744 13779 leader_election.cc:288] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30f55524dbe542db967e61733460bfae (127.8.62.131:45707), 65e3eabcc0ca432fb30659d0cd11bcac (127.8.62.130:35459)
I20260807 18:06:17.339105 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.335358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:60511 (local address 127.8.62.131:45707)
0807 18:06:17.335581 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.335587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.336845 (+ 1258us) server_negotiation.cc:407] Connection header received
0807 18:06:17.337051 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.337057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.337116 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.337227 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.338208 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.338217 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.338226 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.338253 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.338301 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.338321 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.338456 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.338682 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.338776 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:06:17.339135 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.335309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35459 (local address 127.8.62.129:56835)
0807 18:06:17.336130 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.336148 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.336329 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.336781 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.336786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.336813 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.337674 (+ 861us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.337679 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.337703 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.337767 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.338171 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.338176 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.338182 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.338333 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.338405 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":478,"threads_started":1}
I20260807 18:06:17.339655 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.335311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:45707 (local address 127.8.62.129:60511)
0807 18:06:17.336725 (+ 1414us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.336740 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:17.336889 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:17.337253 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:17.337257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:17.337280 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:17.337948 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:06:17.337951 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:17.337969 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:06:17.338020 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:17.338446 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:17.338453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:17.338461 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 18:06:17.338789 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 18:06:17.338903 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"mutex_wait_us":34,"spinlock_wait_cycles":25856,"thread_start_us":938,"threads_started":1}
I20260807 18:06:17.339812 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.335392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:56835 (local address 127.8.62.130:35459)
0807 18:06:17.335580 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.335587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.336260 (+ 673us) server_negotiation.cc:407] Connection header received
0807 18:06:17.336500 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.336508 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.336628 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.336748 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.337950 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.337957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.337962 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.337983 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.338022 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.338038 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.339366 (+ 1328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.339429 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.339504 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:06:17.340865 13708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74410ef98f704446ba1f2ea858bbb149" candidate_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f55524dbe542db967e61733460bfae" is_pre_election: true
I20260807 18:06:17.340878 13576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74410ef98f704446ba1f2ea858bbb149" candidate_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" is_pre_election: true
I20260807 18:06:17.341416 13708 raft_consensus.cc:2454] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef79536310c54a9e8d9edebbfa0d5b30 in term 0.
I20260807 18:06:17.341437 13576 raft_consensus.cc:2454] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef79536310c54a9e8d9edebbfa0d5b30 in term 0.
I20260807 18:06:17.342198 13377 leader_election.cc:302] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 30f55524dbe542db967e61733460bfae, 65e3eabcc0ca432fb30659d0cd11bcac, ef79536310c54a9e8d9edebbfa0d5b30; no voters:
I20260807 18:06:17.342693 13779 raft_consensus.cc:2781] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:17.342825 13779 raft_consensus.cc:482] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:17.342904 13779 raft_consensus.cc:3037] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:17.345743 13779 raft_consensus.cc:504] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.346477 13779 leader_election.cc:288] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [CANDIDATE]: Term 1 election: Requested vote from peers 30f55524dbe542db967e61733460bfae (127.8.62.131:45707), 65e3eabcc0ca432fb30659d0cd11bcac (127.8.62.130:35459)
I20260807 18:06:17.347047 13708 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74410ef98f704446ba1f2ea858bbb149" candidate_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30f55524dbe542db967e61733460bfae"
I20260807 18:06:17.347303 13708 raft_consensus.cc:3037] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:17.347326 13576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74410ef98f704446ba1f2ea858bbb149" candidate_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65e3eabcc0ca432fb30659d0cd11bcac"
I20260807 18:06:17.347673 13576 raft_consensus.cc:3037] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:17.349859 13708 raft_consensus.cc:2454] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef79536310c54a9e8d9edebbfa0d5b30 in term 1.
I20260807 18:06:17.350224 13576 raft_consensus.cc:2454] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef79536310c54a9e8d9edebbfa0d5b30 in term 1.
I20260807 18:06:17.350348 13377 leader_election.cc:302] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30f55524dbe542db967e61733460bfae, ef79536310c54a9e8d9edebbfa0d5b30; no voters:
I20260807 18:06:17.350776 13779 raft_consensus.cc:2781] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:17.351291 13779 raft_consensus.cc:686] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [term 1 LEADER]: Becoming Leader. State: Replica: ef79536310c54a9e8d9edebbfa0d5b30, State: Running, Role: LEADER
I20260807 18:06:17.351822 13779 consensus_queue.cc:237] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } }
I20260807 18:06:17.357201 13300 catalog_manager.cc:5495] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef79536310c54a9e8d9edebbfa0d5b30 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef79536310c54a9e8d9edebbfa0d5b30" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 38045 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 } health_report { overall_health: UNKNOWN } } }
I20260807 18:06:17.398337 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.395933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.8.62.129:38045)
0807 18:06:17.396128 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.396133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.396154 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:17.396404 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.396409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.396625 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.396729 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.397705 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.397710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.397713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.397731 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.397767 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.397780 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.397897 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.398018 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.398074 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:06:17.407624 13576 raft_consensus.cc:1261] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 1 FOLLOWER]: Refusing update from remote peer ef79536310c54a9e8d9edebbfa0d5b30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:17.407658 13708 raft_consensus.cc:1261] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 1 FOLLOWER]: Refusing update from remote peer ef79536310c54a9e8d9edebbfa0d5b30: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 18:06:17.408869 13754 tablet.cc:2369] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:17.408915 13783 consensus_queue.cc:1035] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65e3eabcc0ca432fb30659d0cd11bcac" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:17.409330 13779 consensus_queue.cc:1035] T 74410ef98f704446ba1f2ea858bbb149 P ef79536310c54a9e8d9edebbfa0d5b30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f55524dbe542db967e61733460bfae" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:17.441906 13298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:17.442229 13298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58574:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:17.447129 13298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de1d74e4075d49a9ac2682df4d3e7755: Sending DeleteTablet for 3 replicas of tablet 74410ef98f704446ba1f2ea858bbb149
I20260807 18:06:17.448719 13688 tablet_service.cc:1514] Processing DeleteTablet for tablet 74410ef98f704446ba1f2ea858bbb149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:17 UTC) from {username='slave'} at 127.0.0.1:34918
I20260807 18:06:17.448755 13423 tablet_service.cc:1514] Processing DeleteTablet for tablet 74410ef98f704446ba1f2ea858bbb149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:17 UTC) from {username='slave'} at 127.0.0.1:47644
I20260807 18:06:17.448881 13556 tablet_service.cc:1514] Processing DeleteTablet for tablet 74410ef98f704446ba1f2ea858bbb149 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:17 UTC) from {username='slave'} at 127.0.0.1:54278
I20260807 18:06:17.452361 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13361
I20260807 18:06:17.462783 13796 tablet_replica.cc:330] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: stopping tablet replica
I20260807 18:06:17.463653 13795 tablet_replica.cc:330] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: stopping tablet replica
I20260807 18:06:17.463694 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.457861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.8.62.129:38045)
0807 18:06:17.458139 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.458149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.458574 (+ 425us) server_negotiation.cc:407] Connection header received
0807 18:06:17.458873 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.458881 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.458978 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.459142 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.462980 (+ 3838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.462985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.462988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.463009 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.463044 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.463057 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.463229 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.463322 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.463392 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 18:06:17.464246 13795 raft_consensus.cc:2229] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:17.464766 13485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:17.464916 13795 raft_consensus.cc:2258] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:17.465214 13796 raft_consensus.cc:2229] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:17.465848 13796 raft_consensus.cc:2258] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:17.468078 13795 ts_tablet_manager.cc:1905] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:17.468571 13796 ts_tablet_manager.cc:1905] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:17.478972 13796 ts_tablet_manager.cc:1918] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:17.479329 13796 log.cc:1196] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/74410ef98f704446ba1f2ea858bbb149
I20260807 18:06:17.480085 13796 ts_tablet_manager.cc:1939] T 74410ef98f704446ba1f2ea858bbb149 P 65e3eabcc0ca432fb30659d0cd11bcac: Deleting consensus metadata
I20260807 18:06:17.479131 13795 ts_tablet_manager.cc:1918] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:17.481668 13795 log.cc:1196] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/74410ef98f704446ba1f2ea858bbb149
I20260807 18:06:17.482710 13795 ts_tablet_manager.cc:1939] T 74410ef98f704446ba1f2ea858bbb149 P 30f55524dbe542db967e61733460bfae: Deleting consensus metadata
I20260807 18:06:17.483428 13286 catalog_manager.cc:4841] TS 65e3eabcc0ca432fb30659d0cd11bcac (127.8.62.130:35459): tablet 74410ef98f704446ba1f2ea858bbb149 (table test-table [id=43e5906f8c874ff1b0b5c7bcbe2f8b78]) successfully deleted
I20260807 18:06:17.484851 13285 catalog_manager.cc:4841] TS 30f55524dbe542db967e61733460bfae (127.8.62.131:45707): tablet 74410ef98f704446ba1f2ea858bbb149 (table test-table [id=43e5906f8c874ff1b0b5c7bcbe2f8b78]) successfully deleted
I20260807 18:06:17.578397 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13361
I20260807 18:06:17.597761 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13494
W20260807 18:06:17.597780 13287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.62.129:38045 (error 108)
I20260807 18:06:17.600270 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.599434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47658)
0807 18:06:17.599732 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.599989 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260807 18:06:17.600497 13287 catalog_manager.cc:4584] TS ef79536310c54a9e8d9edebbfa0d5b30 (127.8.62.129:38045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 74410ef98f704446ba1f2ea858bbb149: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
I20260807 18:06:17.601272 13770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.598647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.8.62.130:35459)
0807 18:06:17.598980 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.598987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.599008 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:17.599153 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.599159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.599253 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.599374 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.600435 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.600441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.600446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.600466 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.600505 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.600520 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.600793 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.600928 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.601013 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201}
I20260807 18:06:17.602021 13616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:17.654580 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.653976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47672)
0807 18:06:17.654258 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.654385 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 18:06:17.656054 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.655532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47674)
0807 18:06:17.655723 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.655822 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:06:17.695444 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13494
I20260807 18:06:17.714056 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13624
I20260807 18:06:17.716481 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.716054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47678)
0807 18:06:17.716241 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.716333 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:06:17.717352 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.714937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.8.62.131:45707)
0807 18:06:17.715128 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:17.715132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:17.715153 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:17.715284 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:17.715288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:17.715367 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:17.715455 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:17.716632 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:17.716663 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:17.716673 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:17.716691 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:17.716722 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:17.716735 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:17.716875 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:17.716980 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:06:17.717044 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:06:17.717672 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.717274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47686)
0807 18:06:17.717423 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.717517 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:06:17.718044 13748 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:17.798444 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.797874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47700)
0807 18:06:17.798140 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.798253 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:06:17.799691 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:17.799193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:38045 (local address 127.0.0.1:47708)
0807 18:06:17.799453 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:17.799543 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260807 18:06:17.808460 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13624
I20260807 18:06:17.826942 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13269
I20260807 18:06:17.828208 13330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:17.937474 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13269
2026-08-07T18:06:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2885 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 18:06:17.981993 8442 mini_oidc.cc:152] Starting JWKS server
I20260807 18:06:17.985157 8442 webserver.cc:460] Webserver started at https://127.0.0.1:32821/ using document root <none> and password file <none>
I20260807 18:06:17.985453 8442 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:06:17.986119 8442 webserver.cc:460] Webserver started at http://127.0.0.1:43121/ using document root <none> and password file <none>
2026-08-07T18:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:17Z Disabled control of system clock
I20260807 18:06:18.017244 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37819
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:36961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:37819
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32821/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/testchainca.cert with env {}
W20260807 18:06:18.369369 13809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:18.369817 13809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:18.369987 13809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:18.377684 13809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:18.377869 13809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:18.377962 13809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:18.378073 13809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:18.378158 13809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:06:18.378252 13809 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:06:18.378331 13809 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32821/jwks.json
I20260807 18:06:18.389176 13809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:37819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32821/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:18.390587 13809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:18.391978 13809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:18.404619 13814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:18.404729 13815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:18.404677 13817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:18.406032 13809 server_base.cc:1034] running on GCE node
I20260807 18:06:18.406965 13809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:18.408515 13809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:18.409847 13809 hybrid_clock.cc:648] HybridClock initialized: now 1786125978409806 us; error 42 us; skew 500 ppm
I20260807 18:06:18.410230 13809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:18.412823 13809 webserver.cc:460] Webserver started at http://127.8.62.190:41815/ using document root <none> and password file <none>
I20260807 18:06:18.413585 13809 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:18.413746 13809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:18.414085 13809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:18.417415 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "95b8402d85744d60b7fe692920adb10d"
format_stamp: "Formatted at 2026-08-07 18:06:18 on dist-test-slave-rnrw"
server_key: "8878c39d6b5d40e73e8abad7499f22d6"
server_key_iv: "53ff6f335c667a10e2e78e97db4614ae"
server_key_version: "encryptionkey@0"
I20260807 18:06:18.418375 13809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "95b8402d85744d60b7fe692920adb10d"
format_stamp: "Formatted at 2026-08-07 18:06:18 on dist-test-slave-rnrw"
server_key: "8878c39d6b5d40e73e8abad7499f22d6"
server_key_iv: "53ff6f335c667a10e2e78e97db4614ae"
server_key_version: "encryptionkey@0"
I20260807 18:06:18.424172 13809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 18:06:18.428162 13823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:18.429857 13809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:18.430083 13809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "95b8402d85744d60b7fe692920adb10d"
format_stamp: "Formatted at 2026-08-07 18:06:18 on dist-test-slave-rnrw"
server_key: "8878c39d6b5d40e73e8abad7499f22d6"
server_key_iv: "53ff6f335c667a10e2e78e97db4614ae"
server_key_version: "encryptionkey@0"
I20260807 18:06:18.430347 13809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:18.455163 13809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:18.456133 13809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:18.456483 13809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:18.476603 13809 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:37819
I20260807 18:06:18.476680 13874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:37819 every 8 connection(s)
I20260807 18:06:18.478091 13809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:18.480304 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13809
I20260807 18:06:18.480543 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:18.481227 8442 external_mini_cluster.cc:1428] Setting key a252e9b741776acd14a090fd63b508fc
I20260807 18:06:18.482515 13875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:18.497732 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:18.483283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.8.62.190:37819)
0807 18:06:18.484379 (+ 1096us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:18.484397 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:18.484455 (+ 58us) server_negotiation.cc:407] Connection header received
0807 18:06:18.486354 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:18.486386 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:18.487084 (+ 698us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:18.487598 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:18.488172 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.490332 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.491812 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.492931 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.493144 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:18.494372 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:18.494385 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:18.494410 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:18.495246 (+ 836us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:18.495354 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:18.495387 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:18.495517 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:18.495798 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 18:06:18.496222 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":337,"threads_started":1}
I20260807 18:06:18.498787 13875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d: Bootstrap starting.
I20260807 18:06:18.502619 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:06:18.503073 13875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:18.504415 13875 log.cc:824] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:18.507809 13875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d: No bootstrap required, opened a new log
I20260807 18:06:18.515596 13875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } }
I20260807 18:06:18.516006 13875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:18.516139 13875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b8402d85744d60b7fe692920adb10d, State: Initialized, Role: FOLLOWER
I20260807 18:06:18.516842 13875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } }
I20260807 18:06:18.517108 13875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:18.517282 13875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:18.517470 13875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:18.519716 13875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } }
I20260807 18:06:18.520308 13875 leader_election.cc:302] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95b8402d85744d60b7fe692920adb10d; no voters:
I20260807 18:06:18.521070 13875 leader_election.cc:288] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:18.521229 13880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:18.521974 13880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [term 1 LEADER]: Becoming Leader. State: Replica: 95b8402d85744d60b7fe692920adb10d, State: Running, Role: LEADER
I20260807 18:06:18.522603 13880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } }
I20260807 18:06:18.523600 13875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:18.525825 13881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95b8402d85744d60b7fe692920adb10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } } }
I20260807 18:06:18.526242 13881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:18.526908 13882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95b8402d85744d60b7fe692920adb10d. Latest consensus state: current_term: 1 leader_uuid: "95b8402d85744d60b7fe692920adb10d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95b8402d85744d60b7fe692920adb10d" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37819 } } }
I20260807 18:06:18.527437 13882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:18.528854 13888 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:18.535001 13888 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:18.545403 13888 catalog_manager.cc:1347] Generated new cluster ID: c6df40a21b704ba68e0610ecac6d532f
I20260807 18:06:18.545547 13888 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:18.569834 13888 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:18.571199 13888 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:18.578424 13888 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95b8402d85744d60b7fe692920adb10d: Generated new TSK 0
I20260807 18:06:18.579329 13888 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:06:18.627846 13905 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 18:06:18.628098 13905 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:06:18.630143 13878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:18.597876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.8.62.190:37819)
0807 18:06:18.598104 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:18.598111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:18.598761 (+ 650us) server_negotiation.cc:407] Connection header received
0807 18:06:18.599184 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:18.599188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:18.599263 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:18.599387 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:06:18.600047 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.600839 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.602978 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.603442 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.603539 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:18.604225 (+ 686us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:06:18.629407 (+ 25182us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:06:18.629832 (+ 425us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33034: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":101}
W20260807 18:06:18.630300 13878 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33034: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 18:06:18.632058 13904 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:18.597674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37819 (local address 127.0.0.1:33034)
0807 18:06:18.598632 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:18.598654 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:18.598858 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:18.599450 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:18.599457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:18.599473 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:06:18.599878 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:18.599890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.600995 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.601001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:18.602817 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:18.602833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:18.603623 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:18.603631 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:18.603821 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:18.603900 (+ 79us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:06:18.629780 (+ 25880us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 18:06:18.630184 (+ 404us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.62.190:37819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":745,"thread_start_us":584,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.62.190:37819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32821/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 18:06:18.634315 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13809
I20260807 18:06:18.635999 13870 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:18.738226 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13809
2026-08-07T18:06:18Z chronyd exiting
I20260807 18:06:18.791401 8442 test_util.cc:175] -----------------------------------------------
I20260807 18:06:18.791572 8442 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786125932226923-8442-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (814 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 18:06:18.795607 8442 mini_oidc.cc:152] Starting JWKS server
I20260807 18:06:18.797600 8442 webserver.cc:460] Webserver started at https://127.0.0.1:43749/ using document root <none> and password file <none>
I20260807 18:06:18.797811 8442 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:06:18.798492 8442 webserver.cc:460] Webserver started at http://127.0.0.1:46425/ using document root <none> and password file <none>
2026-08-07T18:06:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:18Z Disabled control of system clock
I20260807 18:06:18.828341 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38849
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:36339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:38849
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43749/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/testchainca.cert with env {}
W20260807 18:06:19.177273 13915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:19.177618 13915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:19.177831 13915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:19.185559 13915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:19.185700 13915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:19.185778 13915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:19.185875 13915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:19.185961 13915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:06:19.186077 13915 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:06:19.186138 13915 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43749/jwks.json
I20260807 18:06:19.196993 13915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:38849
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43749/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:19.198640 13915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:19.200217 13915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:19.213276 13920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:19.213320 13921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:19.213234 13923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:19.214638 13915 server_base.cc:1034] running on GCE node
I20260807 18:06:19.215512 13915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:19.217087 13915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:19.218338 13915 hybrid_clock.cc:648] HybridClock initialized: now 1786125979218300 us; error 41 us; skew 500 ppm
I20260807 18:06:19.218734 13915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:19.221345 13915 webserver.cc:460] Webserver started at http://127.8.62.190:34409/ using document root <none> and password file <none>
I20260807 18:06:19.222113 13915 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:19.222260 13915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:19.222621 13915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:19.225767 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "aca3db22497e42fda14411ef7d462ce3"
format_stamp: "Formatted at 2026-08-07 18:06:19 on dist-test-slave-rnrw"
server_key: "2b7f89ffbb1b1bc469783efb8ff7a48e"
server_key_iv: "6ec165b651243baa9d1e82ecb7ea0269"
server_key_version: "encryptionkey@0"
I20260807 18:06:19.226706 13915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "aca3db22497e42fda14411ef7d462ce3"
format_stamp: "Formatted at 2026-08-07 18:06:19 on dist-test-slave-rnrw"
server_key: "2b7f89ffbb1b1bc469783efb8ff7a48e"
server_key_iv: "6ec165b651243baa9d1e82ecb7ea0269"
server_key_version: "encryptionkey@0"
I20260807 18:06:19.232352 13915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:19.236742 13929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:19.238389 13915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:19.238636 13915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "aca3db22497e42fda14411ef7d462ce3"
format_stamp: "Formatted at 2026-08-07 18:06:19 on dist-test-slave-rnrw"
server_key: "2b7f89ffbb1b1bc469783efb8ff7a48e"
server_key_iv: "6ec165b651243baa9d1e82ecb7ea0269"
server_key_version: "encryptionkey@0"
I20260807 18:06:19.238906 13915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:19.256021 13915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:19.257092 13915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:19.257493 13915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:19.278658 13915 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:38849
I20260807 18:06:19.278678 13982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:38849 every 8 connection(s)
I20260807 18:06:19.280226 13915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:19.280963 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 13915
I20260807 18:06:19.281206 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:19.282055 8442 external_mini_cluster.cc:1428] Setting key 0155a3d5913131ee435214d1a5dd8ea4
I20260807 18:06:19.285228 13983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:19.299155 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:19.284223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.8.62.190:38849)
0807 18:06:19.285386 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:19.285403 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:19.285462 (+ 59us) server_negotiation.cc:407] Connection header received
0807 18:06:19.287832 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:19.287864 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:19.288588 (+ 724us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:19.289118 (+ 530us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:19.289738 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.291912 (+ 2174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.293403 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.294492 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.294705 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:19.295885 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:19.295901 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:19.295920 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:19.296763 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:19.296881 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:19.296915 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:19.297040 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:19.297304 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 18:06:19.297709 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":384,"threads_started":1}
I20260807 18:06:19.300760 13983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3: Bootstrap starting.
I20260807 18:06:19.303773 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:06:19.304826 13983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:19.305958 13983 log.cc:824] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:19.309296 13983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3: No bootstrap required, opened a new log
I20260807 18:06:19.317023 13983 raft_consensus.cc:348] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } }
I20260807 18:06:19.317476 13983 raft_consensus.cc:374] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:19.317595 13983 raft_consensus.cc:729] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aca3db22497e42fda14411ef7d462ce3, State: Initialized, Role: FOLLOWER
I20260807 18:06:19.318217 13983 consensus_queue.cc:260] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } }
I20260807 18:06:19.318495 13983 raft_consensus.cc:388] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:19.318650 13983 raft_consensus.cc:482] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:19.318840 13983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:19.321264 13983 raft_consensus.cc:504] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } }
I20260807 18:06:19.321842 13983 leader_election.cc:302] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aca3db22497e42fda14411ef7d462ce3; no voters:
I20260807 18:06:19.322482 13983 leader_election.cc:288] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:19.322692 13988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:19.323899 13988 raft_consensus.cc:686] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [term 1 LEADER]: Becoming Leader. State: Replica: aca3db22497e42fda14411ef7d462ce3, State: Running, Role: LEADER
I20260807 18:06:19.324503 13988 consensus_queue.cc:237] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } }
I20260807 18:06:19.325053 13983 sys_catalog.cc:564] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:19.327739 13989 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aca3db22497e42fda14411ef7d462ce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } } }
I20260807 18:06:19.328248 13989 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:19.328122 13990 sys_catalog.cc:455] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aca3db22497e42fda14411ef7d462ce3. Latest consensus state: current_term: 1 leader_uuid: "aca3db22497e42fda14411ef7d462ce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aca3db22497e42fda14411ef7d462ce3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38849 } } }
I20260807 18:06:19.328420 13990 sys_catalog.cc:458] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:19.330947 13997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:19.336725 13997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:19.346127 13997 catalog_manager.cc:1347] Generated new cluster ID: 5ae114762acc44129f115f5f34997895
I20260807 18:06:19.346266 13997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:19.365463 13997 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:19.367204 13997 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:19.381054 13997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aca3db22497e42fda14411ef7d462ce3: Generated new TSK 0
I20260807 18:06:19.382085 13997 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 18:06:19.436982 14013 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 18:06:19.437134 14013 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:06:19.438746 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:19.410307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38500 (local address 127.8.62.190:38849)
0807 18:06:19.410524 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:19.410532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:19.411001 (+ 469us) server_negotiation.cc:407] Connection header received
0807 18:06:19.411321 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:19.411326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:19.411404 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:19.411584 (+ 180us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:06:19.412122 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.412991 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.414460 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.414954 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.415161 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:19.415755 (+ 594us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:06:19.437595 (+ 21840us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:06:19.438374 (+ 779us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38500: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 18:06:19.438911 13986 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38500: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 18:06:19.438962 14012 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:19.410183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38849 (local address 127.0.0.1:38500)
0807 18:06:19.410875 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:19.410892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:19.411083 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:19.411619 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:19.411624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:19.411640 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:06:19.411920 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:19.411930 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.413156 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.413169 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:19.414288 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:19.414300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:19.415190 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:19.415197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:19.415386 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:19.415425 (+ 39us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:06:19.438130 (+ 22705us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 18:06:19.438462 (+ 332us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.62.190:38849: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43749/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":371,"threads_started":1}
I20260807 18:06:19.440644 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 13915
I20260807 18:06:19.442024 13978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:19.541659 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 13915
2026-08-07T18:06:19Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (804 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 18:06:19.599975 8442 mini_oidc.cc:152] Starting JWKS server
I20260807 18:06:19.602022 8442 webserver.cc:460] Webserver started at https://127.0.0.1:37655/ using document root <none> and password file <none>
I20260807 18:06:19.602315 8442 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 18:06:19.603001 8442 webserver.cc:460] Webserver started at http://127.0.0.1:44825/ using document root <none> and password file <none>
2026-08-07T18:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:19Z Disabled control of system clock
I20260807 18:06:19.633422 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40621
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:34225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:40621
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37655/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:06:19.982220 14020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:19.982647 14020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:19.982816 14020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:19.990357 14020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:19.990608 14020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:19.990713 14020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:19.990805 14020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:19.990895 14020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:06:19.990989 14020 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:06:19.991066 14020 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37655/jwks.json
I20260807 18:06:20.000767 14020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:34225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:40621
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37655/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:20.002206 14020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:20.004199 14020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:20.017093 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:20.017045 14026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:20.017015 14028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:20.017988 14020 server_base.cc:1034] running on GCE node
I20260807 18:06:20.018831 14020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:20.020354 14020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:20.021597 14020 hybrid_clock.cc:648] HybridClock initialized: now 1786125980021543 us; error 44 us; skew 500 ppm
I20260807 18:06:20.021946 14020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:20.024466 14020 webserver.cc:460] Webserver started at http://127.8.62.190:39005/ using document root <none> and password file <none>
I20260807 18:06:20.025238 14020 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:20.025331 14020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:20.025619 14020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:20.028867 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "4aa838abdd3a440197ec720f5c136909"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "bc62676352abf5abc3028e47212e2a8a"
server_key_iv: "4e0109618e45f7fdf812c186d6dd081c"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.029734 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "4aa838abdd3a440197ec720f5c136909"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "bc62676352abf5abc3028e47212e2a8a"
server_key_iv: "4e0109618e45f7fdf812c186d6dd081c"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.035823 14020 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:06:20.039768 14034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:20.041594 14020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:06:20.041755 14020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "4aa838abdd3a440197ec720f5c136909"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "bc62676352abf5abc3028e47212e2a8a"
server_key_iv: "4e0109618e45f7fdf812c186d6dd081c"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.042008 14020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:20.054898 14020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:20.055833 14020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:20.056123 14020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:20.076239 14020 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:40621
I20260807 18:06:20.076256 14087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:40621 every 8 connection(s)
I20260807 18:06:20.078312 14020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:20.083144 14088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:20.086154 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14020
I20260807 18:06:20.086388 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:20.087059 8442 external_mini_cluster.cc:1428] Setting key 96484d497881df81e928a46d0b0400a0
I20260807 18:06:20.098404 14088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909: Bootstrap starting.
I20260807 18:06:20.103145 14088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:20.104719 14088 log.cc:824] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:20.105664 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:20.089400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.8.62.190:40621)
0807 18:06:20.090758 (+ 1358us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:20.090776 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:20.090838 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:06:20.093312 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:20.093352 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:20.094119 (+ 767us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:20.094724 (+ 605us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:20.095442 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.097551 (+ 2109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.099275 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.100496 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.100794 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.102139 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:20.102151 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:20.102174 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:20.103034 (+ 860us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:20.103195 (+ 161us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:20.103228 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:20.103348 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:20.103719 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 18:06:20.104115 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":374,"threads_started":1}
I20260807 18:06:20.108551 14088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909: No bootstrap required, opened a new log
I20260807 18:06:20.110543 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:06:20.118225 14088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } }
I20260807 18:06:20.118639 14088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:20.118732 14088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa838abdd3a440197ec720f5c136909, State: Initialized, Role: FOLLOWER
I20260807 18:06:20.119380 14088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } }
I20260807 18:06:20.119616 14088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:20.119757 14088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:20.119911 14088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:20.122634 14088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } }
I20260807 18:06:20.123215 14088 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa838abdd3a440197ec720f5c136909; no voters:
I20260807 18:06:20.123845 14088 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:20.124027 14093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:20.124868 14093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa838abdd3a440197ec720f5c136909, State: Running, Role: LEADER
I20260807 18:06:20.125492 14093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } }
I20260807 18:06:20.126168 14088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:20.129403 14095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa838abdd3a440197ec720f5c136909. Latest consensus state: current_term: 1 leader_uuid: "4aa838abdd3a440197ec720f5c136909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } } }
I20260807 18:06:20.129848 14095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:20.130679 14094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa838abdd3a440197ec720f5c136909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa838abdd3a440197ec720f5c136909" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40621 } } }
I20260807 18:06:20.131022 14094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:20.132659 14102 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:20.139192 14102 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:20.148954 14102 catalog_manager.cc:1347] Generated new cluster ID: 8ede9a4298cf433e8d60220b64fd0d9a
I20260807 18:06:20.149109 14102 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:20.157974 14102 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:20.159422 14102 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:20.170058 14102 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa838abdd3a440197ec720f5c136909: Generated new TSK 0
I20260807 18:06:20.171290 14102 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:06:20.205078 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:20.195738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.8.62.190:40621)
0807 18:06:20.196044 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:20.196053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:20.196439 (+ 386us) server_negotiation.cc:407] Connection header received
0807 18:06:20.196769 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:20.196774 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:20.196856 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:20.197005 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:06:20.197599 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.198385 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.200644 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.201125 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.201235 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.201989 (+ 754us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:06:20.204230 (+ 2241us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37655/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:06:20.204711 (+ 481us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37655/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 18:06:20.205221 14117 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:20.195625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40621 (local address 127.0.0.1:58154)
0807 18:06:20.196308 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:20.196329 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:20.196509 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:20.197059 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:20.197065 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:20.197079 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:06:20.197399 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:20.197411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.198567 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.198574 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:20.200369 (+ 1795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:20.200388 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.201366 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.201374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:20.201589 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.201647 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:06:20.204464 (+ 2817us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37655/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 18:06:20.204784 (+ 320us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.62.190:40621: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37655/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":353,"threads_started":1}
W20260807 18:06:20.205230 14091 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58154: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37655/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260807 18:06:20.206539 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14020
I20260807 18:06:20.207829 14083 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 18:06:20.216495 14118 webserver.cc:533] Webserver: sslize failed:
E20260807 18:06:20.216677 14118 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 18:06:20.312984 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14020
2026-08-07T18:06:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (771 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T18:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:20Z Disabled control of system clock
I20260807 18:06:20.402350 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35605
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:37787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35605
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/testchainca.cert with env {}
W20260807 18:06:20.751488 14123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:20.751879 14123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:20.752056 14123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:20.760123 14123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:20.760272 14123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:20.760352 14123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:20.760457 14123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:20.760573 14123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 18:06:20.760706 14123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 18:06:20.760766 14123 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 18:06:20.771733 14123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:20.773366 14123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:20.774904 14123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:20.792575 14128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:20.792508 14129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:20.792533 14131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:20.793821 14123 server_base.cc:1034] running on GCE node
I20260807 18:06:20.794754 14123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:20.796487 14123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:20.797813 14123 hybrid_clock.cc:648] HybridClock initialized: now 1786125980797761 us; error 46 us; skew 500 ppm
I20260807 18:06:20.798271 14123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:20.805860 14123 webserver.cc:460] Webserver started at http://127.8.62.190:45219/ using document root <none> and password file <none>
I20260807 18:06:20.806777 14123 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:20.806916 14123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:20.807372 14123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:20.811167 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "0fbc4c2ba52148ae83542edecf981e71"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "155931204e394341baa24e41c26031f7"
server_key_iv: "d707325483c4371dd7b16026c693d1e3"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.812122 14123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "0fbc4c2ba52148ae83542edecf981e71"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "155931204e394341baa24e41c26031f7"
server_key_iv: "d707325483c4371dd7b16026c693d1e3"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.818148 14123 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:06:20.822357 14138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:20.824098 14123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:20.824337 14123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "0fbc4c2ba52148ae83542edecf981e71"
format_stamp: "Formatted at 2026-08-07 18:06:20 on dist-test-slave-rnrw"
server_key: "155931204e394341baa24e41c26031f7"
server_key_iv: "d707325483c4371dd7b16026c693d1e3"
server_key_version: "encryptionkey@0"
I20260807 18:06:20.824640 14123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:20.843080 14123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:20.844055 14123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:20.844381 14123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:20.865772 14123 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:35605
I20260807 18:06:20.865792 14190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:35605 every 8 connection(s)
I20260807 18:06:20.867319 14123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:20.871990 14191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:20.876225 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14123
I20260807 18:06:20.876451 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:20.877382 8442 external_mini_cluster.cc:1428] Setting key 3f731b0a6413696b9088646be84a1bdd
I20260807 18:06:20.886353 14191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71: Bootstrap starting.
I20260807 18:06:20.889892 14191 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:20.891239 14191 log.cc:824] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:20.892396 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:20.879574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.8.62.190:35605)
0807 18:06:20.880769 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:20.880782 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:20.880832 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:06:20.882572 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:20.882599 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:20.883181 (+ 582us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:20.883684 (+ 503us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:20.884406 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.885771 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.887323 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.888097 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.888342 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.889648 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:20.889656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:20.889671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:06:20.890346 (+ 675us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:20.890464 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:06:20.890489 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:20.890630 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:20.890900 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 18:06:20.891260 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":401,"threads_started":1}
I20260807 18:06:20.894930 14191 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71: No bootstrap required, opened a new log
I20260807 18:06:20.897480 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:06:20.904533 14191 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } }
I20260807 18:06:20.905073 14191 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:20.905228 14191 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fbc4c2ba52148ae83542edecf981e71, State: Initialized, Role: FOLLOWER
I20260807 18:06:20.905987 14191 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } }
I20260807 18:06:20.906275 14191 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:20.906441 14191 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:20.906646 14191 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:20.909206 14191 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } }
I20260807 18:06:20.910043 14191 leader_election.cc:302] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fbc4c2ba52148ae83542edecf981e71; no voters:
I20260807 18:06:20.910781 14191 leader_election.cc:288] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:20.911022 14196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:20.911856 14196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [term 1 LEADER]: Becoming Leader. State: Replica: 0fbc4c2ba52148ae83542edecf981e71, State: Running, Role: LEADER
I20260807 18:06:20.912496 14196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } }
I20260807 18:06:20.913302 14191 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:20.916393 14198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fbc4c2ba52148ae83542edecf981e71. Latest consensus state: current_term: 1 leader_uuid: "0fbc4c2ba52148ae83542edecf981e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } }
I20260807 18:06:20.917972 14198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:20.916745 14197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fbc4c2ba52148ae83542edecf981e71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fbc4c2ba52148ae83542edecf981e71" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } }
I20260807 18:06:20.919095 14197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:20.920603 14205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:20.926775 14205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:20.937211 14205 catalog_manager.cc:1347] Generated new cluster ID: 83d65e1a87d345c7b3a55c928b99b257
I20260807 18:06:20.937355 14205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:20.955776 14205 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:20.957749 14205 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:20.966925 14205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fbc4c2ba52148ae83542edecf981e71: Generated new TSK 0
I20260807 18:06:20.967939 14205 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:06:21.453945 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:20.992670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60446 (local address 127.8.62.190:35605)
0807 18:06:20.992885 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:20.992892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:20.993301 (+ 409us) server_negotiation.cc:407] Connection header received
0807 18:06:20.993585 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:20.993592 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:20.993693 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:20.993905 (+ 212us) server_negotiation.cc:226] Negotiated authn=JWT
0807 18:06:20.994477 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.995687 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.997274 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.997988 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.998123 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.998631 (+ 508us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 18:06:21.453359 (+454728us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:06:21.453716 (+ 357us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60446: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 18:06:21.454072 14194 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60446: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 18:06:21.454432 14220 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:20.992506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35605 (local address 127.0.0.1:60446)
0807 18:06:20.993195 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:20.993212 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:20.993341 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:20.993968 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:20.993972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:20.993985 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0807 18:06:20.994303 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:20.994312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.995861 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.995865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:20.997091 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:20.997129 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:20.998182 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:20.998186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:20.998307 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:20.998345 (+ 38us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 18:06:21.453702 (+455357us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 18:06:21.453969 (+ 267us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.62.190:35605: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":348,"threads_started":1}
I20260807 18:06:21.455955 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14123
I20260807 18:06:21.457234 14186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:21.561424 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14123
2026-08-07T18:06:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1229 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (457 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (445 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:22 dist-test-slave-rnrw krb5kdc[14235](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:22 dist-test-slave-rnrw krb5kdc[14235](info): set up 2 sockets
Aug 07 18:06:22 dist-test-slave-rnrw krb5kdc[14235](info): commencing operation
krb5kdc: starting...
W20260807 18:06:23.608404 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:23 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:23.857607 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38685
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:38685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:24.214972 14248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:24.215310 14248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:24.215479 14248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:24.223192 14248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:24.223335 14248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:24.223425 14248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:24.223523 14248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:24.223615 14248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:24.234154 14248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:38685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:24.235711 14248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:24.237254 14248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:24.251221 14253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:24.251315 14254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:24.251286 14256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:24.252537 14248 server_base.cc:1034] running on GCE node
I20260807 18:06:24.253576 14248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:24.255420 14248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:24.256721 14248 hybrid_clock.cc:648] HybridClock initialized: now 1786125984256664 us; error 37 us; skew 500 ppm
I20260807 18:06:24.257143 14248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:24 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125984, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:24.260840 14248 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:24.262717 14248 webserver.cc:460] Webserver started at http://127.8.62.190:44231/ using document root <none> and password file <none>
I20260807 18:06:24.263468 14248 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:24.263584 14248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:24.263882 14248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:24.267271 14248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "42a964298d4647ce99c5774f55f64a67"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "2188396bd9c428c6befa788d85ba74fd"
server_key_iv: "c89f09114b898832af83624aaec3db76"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.268111 14248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "42a964298d4647ce99c5774f55f64a67"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "2188396bd9c428c6befa788d85ba74fd"
server_key_iv: "c89f09114b898832af83624aaec3db76"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.274044 14248 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:06:24.278075 14263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:24.279825 14248 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:24.280148 14248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "42a964298d4647ce99c5774f55f64a67"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "2188396bd9c428c6befa788d85ba74fd"
server_key_iv: "c89f09114b898832af83624aaec3db76"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.280369 14248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:24.299211 14248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:24.304402 14248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:24.304868 14248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:24.325848 14248 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:38685
I20260807 18:06:24.325881 14315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:38685 every 8 connection(s)
I20260807 18:06:24.327544 14248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:24.331843 14316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:24.332523 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14248
I20260807 18:06:24.332883 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:24.333568 8442 external_mini_cluster.cc:1428] Setting key 0ba21341f3ee02ec94d052a7af905ed7
I20260807 18:06:24.348294 14316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67: Bootstrap starting.
Aug 07 18:06:24 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:24.352418 14316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:24.353641 14316 log.cc:824] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:24.356926 14316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67: No bootstrap required, opened a new log
I20260807 18:06:24.361717 14319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:24.336032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.8.62.190:38685)
0807 18:06:24.337053 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:24.337067 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:24.337119 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:06:24.339278 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:24.339316 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:24.339861 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:24.340295 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:24.342299 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.343807 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.345812 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.346878 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.347085 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:24.352177 (+ 5092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:24.352187 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:24.352205 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:24.352252 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:24.356330 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:24.357258 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:24.357266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:24.357274 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:24.357473 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:24.357934 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:24.357938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:24.357940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:24.358569 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:24.358997 (+ 428us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:24.359536 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 18:06:24.360044 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":487,"threads_started":1}
I20260807 18:06:24.366767 14316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } }
I20260807 18:06:24.367321 14316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:24.367478 14316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a964298d4647ce99c5774f55f64a67, State: Initialized, Role: FOLLOWER
I20260807 18:06:24.368196 14316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } }
I20260807 18:06:24.368496 14316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:24.368672 14316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:24.368822 14316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:24.371351 14316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } }
I20260807 18:06:24.372014 14316 leader_election.cc:302] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a964298d4647ce99c5774f55f64a67; no voters:
I20260807 18:06:24.372792 14316 leader_election.cc:288] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:24.372913 14321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:24.373662 14321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [term 1 LEADER]: Becoming Leader. State: Replica: 42a964298d4647ce99c5774f55f64a67, State: Running, Role: LEADER
I20260807 18:06:24.374296 14321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } }
I20260807 18:06:24.375129 14316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:24.377570 14323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a964298d4647ce99c5774f55f64a67. Latest consensus state: current_term: 1 leader_uuid: "42a964298d4647ce99c5774f55f64a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } } }
I20260807 18:06:24.377899 14323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:24.377432 14322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42a964298d4647ce99c5774f55f64a67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a964298d4647ce99c5774f55f64a67" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38685 } } }
I20260807 18:06:24.378396 14322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:24.380407 14328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:24.386215 14328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:24.396420 14328 catalog_manager.cc:1347] Generated new cluster ID: 58856397ad584160b523c44776bb2ad8
I20260807 18:06:24.396596 14328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:24.417809 14328 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:24.419912 14328 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:24.430716 14328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42a964298d4647ce99c5774f55f64a67: Generated new TSK 0
I20260807 18:06:24.431722 14328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:06:24.500742 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:24.847970 14342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:24.848276 14342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:24.848472 14342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:24.856233 14342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:24.856360 14342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:24.856689 14342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:24.867837 14342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:24.869377 14342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:24.870922 14342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:24.886416 14348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:24.886315 14350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:24.886335 14347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:24.887511 14342 server_base.cc:1034] running on GCE node
I20260807 18:06:24.888231 14342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:24.889410 14342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:24.890726 14342 hybrid_clock.cc:648] HybridClock initialized: now 1786125984890680 us; error 46 us; skew 500 ppm
I20260807 18:06:24.891192 14342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:24 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125984, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:24.895112 14342 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:06:24.896975 14342 webserver.cc:460] Webserver started at http://127.8.62.129:44601/ using document root <none> and password file <none>
I20260807 18:06:24.897764 14342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:24.897881 14342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:24.898159 14342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:24.901535 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "57ac009c87064c9fa515e19ad4730416"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "b6824d39374f9660c3a1615222af4836"
server_key_iv: "e092192818a787328f129f1eaf48bb6c"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.902412 14342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "57ac009c87064c9fa515e19ad4730416"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "b6824d39374f9660c3a1615222af4836"
server_key_iv: "e092192818a787328f129f1eaf48bb6c"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.908746 14342 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 18:06:24.913043 14357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:24.914805 14342 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:06:24.915023 14342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "57ac009c87064c9fa515e19ad4730416"
format_stamp: "Formatted at 2026-08-07 18:06:24 on dist-test-slave-rnrw"
server_key: "b6824d39374f9660c3a1615222af4836"
server_key_iv: "e092192818a787328f129f1eaf48bb6c"
server_key_version: "encryptionkey@0"
I20260807 18:06:24.915283 14342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:24.933300 14342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:24.937789 14342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:24.938117 14342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:24.939170 14342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:24.940680 14342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:24.940788 14342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:24.940944 14342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:24.941021 14342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:24.976094 14342 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:38603
I20260807 18:06:24.976159 14470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:38603 every 8 connection(s)
I20260807 18:06:24.977608 14342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:24.984261 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14342
I20260807 18:06:24.984529 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:24.985343 8442 external_mini_cluster.cc:1428] Setting key 9ca867131d65bc4ae98b4b780885621c
Aug 07 18:06:24 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125984, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:24.999905 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:24.980536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:51697 (local address 127.8.62.190:38685)
0807 18:06:24.981117 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:24.981121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:24.981942 (+ 821us) server_negotiation.cc:407] Connection header received
0807 18:06:24.983920 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:24.983926 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:24.984039 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:24.984178 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:24.986248 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.987088 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.988876 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.989316 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.989500 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:24.993441 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:24.993447 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:24.993450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:24.993489 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:24.996378 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:24.997352 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:24.997363 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:24.997366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:24.997504 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:24.997987 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:24.997995 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:24.998002 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:24.998286 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:24.998469 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:24.999252 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 18:06:24.999521 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":332,"threads_started":1}
I20260807 18:06:25.001049 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:24.980940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38685 (local address 127.8.62.129:51697)
0807 18:06:24.981760 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:24.981810 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:24.983595 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:24.984497 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:24.984510 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:24.985314 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:24.986036 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:24.986069 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.987275 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.987281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:24.988717 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:24.988732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:24.989629 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:24.989637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:24.989892 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:24.990687 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:24.990712 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:24.993129 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:24.996620 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:24.996631 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:24.996651 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:24.997192 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:24.997635 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:24.997640 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:24.997644 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:24.997851 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:24.998430 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:24.998439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:24.998827 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:06:24.999129 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 18:06:24.999719 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":320,"threads_started":1}
I20260807 18:06:25.002924 14471 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38685
I20260807 18:06:25.003324 14471 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:25.004392 14471 heartbeater.cc:511] Master 127.8.62.190:38685 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:25.007025 14280 ts_manager.cc:194] Registered new tserver with Master: 57ac009c87064c9fa515e19ad4730416 (127.8.62.129:38603)
I20260807 18:06:25.008913 14280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:51697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:06:25.037231 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:25.383769 14477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:25.384080 14477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:25.384178 14477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:25.392073 14477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:25.392222 14477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:25.392341 14477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:25.402697 14477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:25.404044 14477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:25.405459 14477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:25.420126 14482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:25.420282 14483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:25.420305 14485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:25.421247 14477 server_base.cc:1034] running on GCE node
I20260807 18:06:25.421972 14477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:25.423079 14477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:25.424346 14477 hybrid_clock.cc:648] HybridClock initialized: now 1786125985424303 us; error 70 us; skew 500 ppm
I20260807 18:06:25.424763 14477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:25 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125985, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:25.428221 14477 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:25.430009 14477 webserver.cc:460] Webserver started at http://127.8.62.130:43149/ using document root <none> and password file <none>
I20260807 18:06:25.430807 14477 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:25.430964 14477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:25.431308 14477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:25.434481 14477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "59d6be63bec94343b8d2914d2241dbf2"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "a84d42cd8ed1881fc270609f68d94bb4"
server_key_iv: "c344564170d9bec98003cb23e3879e98"
server_key_version: "encryptionkey@0"
I20260807 18:06:25.435341 14477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "59d6be63bec94343b8d2914d2241dbf2"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "a84d42cd8ed1881fc270609f68d94bb4"
server_key_iv: "c344564170d9bec98003cb23e3879e98"
server_key_version: "encryptionkey@0"
I20260807 18:06:25.441174 14477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:06:25.445118 14492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:25.446720 14477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:06:25.446939 14477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "59d6be63bec94343b8d2914d2241dbf2"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "a84d42cd8ed1881fc270609f68d94bb4"
server_key_iv: "c344564170d9bec98003cb23e3879e98"
server_key_version: "encryptionkey@0"
I20260807 18:06:25.447188 14477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:25.495050 14477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:25.499693 14477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:25.500061 14477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:25.501308 14477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:25.502822 14477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:25.502936 14477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:25.503094 14477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:25.503178 14477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:25.539009 14477 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:45023
I20260807 18:06:25.539076 14606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:45023 every 8 connection(s)
I20260807 18:06:25.540426 14477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:25.540918 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14477
I20260807 18:06:25.541119 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:25.541697 8442 external_mini_cluster.cc:1428] Setting key 826768e7a4fba235e85a4ab542f3619e
Aug 07 18:06:25 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125985, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:06:25.568337 14609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:25.543634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45289 (local address 127.8.62.190:38685)
0807 18:06:25.544250 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:25.544256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:25.545574 (+ 1318us) server_negotiation.cc:407] Connection header received
0807 18:06:25.548673 (+ 3099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:25.548678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:25.548754 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:25.548866 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:25.551359 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:25.552309 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:25.555050 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:25.555545 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:25.555670 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:25.560338 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:25.560347 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:25.560355 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:25.560407 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:25.564516 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:25.565476 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:25.565486 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:25.565492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:25.565694 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:25.566280 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:25.566288 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:25.566292 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:25.566741 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:25.566920 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:25.567812 (+ 892us) server_negotiation.cc:299] Negotiation successful
0807 18:06:25.567999 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":387,"threads_started":1}
I20260807 18:06:25.569437 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:25.544334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38685 (local address 127.8.62.130:45289)
0807 18:06:25.545370 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:25.545430 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:25.548224 (+ 2794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:25.549199 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:25.549214 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:25.550089 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:25.551138 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:25.551172 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:25.552462 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:25.552470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:25.554864 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:25.554885 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:25.555822 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:25.555832 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:25.556213 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:25.557027 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:25.557064 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:25.560013 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:25.564766 (+ 4753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:25.564775 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:25.564791 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:25.565301 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:25.565861 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:25.565866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:25.565868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:25.566116 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:25.566903 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:25.566911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:25.567352 (+ 441us) client_negotiation.cc:769] Sending connection context
0807 18:06:25.567691 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 18:06:25.568151 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":485,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:06:25.571171 14607 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38685
I20260807 18:06:25.571556 14607 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:25.572633 14607 heartbeater.cc:511] Master 127.8.62.190:38685 requested a full tablet report, sending...
I20260807 18:06:25.574421 14280 ts_manager.cc:194] Registered new tserver with Master: 59d6be63bec94343b8d2914d2241dbf2 (127.8.62.130:45023)
I20260807 18:06:25.575403 14280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:45289
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:25.601577 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:25.941268 14613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:25.941565 14613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:25.941735 14613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:25.949110 14613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:25.949230 14613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:25.949426 14613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:25.959671 14613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38685
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:25.961035 14613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:25.962387 14613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:25.975920 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:25.976114 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:25.976624 14620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:25.976926 14613 server_base.cc:1034] running on GCE node
I20260807 18:06:25.977623 14613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:25.980355 14613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:25.981592 14613 hybrid_clock.cc:648] HybridClock initialized: now 1786125985981527 us; error 60 us; skew 500 ppm
I20260807 18:06:25.981937 14613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:25 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125985, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:25.989454 14613 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:25.991197 14613 webserver.cc:460] Webserver started at http://127.8.62.131:46101/ using document root <none> and password file <none>
I20260807 18:06:25.992014 14613 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:25.992170 14613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:25.992522 14613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:25.995785 14613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "0142060f80d7453c9d783d405d62242d"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "b5d2e966c899998db085e3ee864e5e51"
server_key_iv: "0fee0a40e26fc0c682c7922b8b021c15"
server_key_version: "encryptionkey@0"
I20260807 18:06:25.996731 14613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "0142060f80d7453c9d783d405d62242d"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "b5d2e966c899998db085e3ee864e5e51"
server_key_iv: "0fee0a40e26fc0c682c7922b8b021c15"
server_key_version: "encryptionkey@0"
I20260807 18:06:26.002583 14613 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:06:26.006776 14629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:26.008481 14613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:06:26.008828 14613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "0142060f80d7453c9d783d405d62242d"
format_stamp: "Formatted at 2026-08-07 18:06:25 on dist-test-slave-rnrw"
server_key: "b5d2e966c899998db085e3ee864e5e51"
server_key_iv: "0fee0a40e26fc0c682c7922b8b021c15"
server_key_version: "encryptionkey@0"
I20260807 18:06:26.009076 14613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:26.012619 14471 heartbeater.cc:503] Master 127.8.62.190:38685 was elected leader, sending a full tablet report...
I20260807 18:06:26.037530 14613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:26.042488 14613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:26.042920 14613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:26.044072 14613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:26.045569 14613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:26.045660 14613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:26.045799 14613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:26.045851 14613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:26.082240 14613 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:41103
I20260807 18:06:26.082288 14742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:41103 every 8 connection(s)
I20260807 18:06:26.083562 14613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:26.084255 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14613
I20260807 18:06:26.084450 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:26.085140 8442 external_mini_cluster.cc:1428] Setting key 9ff8c34ce2b3b3a79aafc9c4ac64747b
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125985, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:26.105759 14745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.086516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:56123 (local address 127.8.62.190:38685)
0807 18:06:26.087366 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.087374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.088283 (+ 909us) server_negotiation.cc:407] Connection header received
0807 18:06:26.090271 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.090275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.090359 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.090469 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.092193 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.093089 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.094686 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.095087 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.095197 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.099146 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.099156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.099163 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.099225 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.102673 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.103498 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.103502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.103504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.103634 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.104069 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.104072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.104074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.104368 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.104488 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.105282 (+ 794us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.105462 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":606,"threads_started":1}
I20260807 18:06:26.106874 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.087198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38685 (local address 127.8.62.131:56123)
0807 18:06:26.088131 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:26.088184 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:26.089958 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:26.090747 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:26.090758 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:26.091358 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:26.091991 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:26.092010 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.093236 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.093240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:26.094526 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:26.094544 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.095294 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.095299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:26.095504 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.096210 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:26.096235 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:26.098832 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:26.102841 (+ 4009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:26.102850 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:26.102866 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:26.103361 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:26.103763 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:26.103766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:26.103769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:26.103975 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:26.104485 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:26.104491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:26.104948 (+ 457us) client_negotiation.cc:769] Sending connection context
0807 18:06:26.105201 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 18:06:26.105635 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":401,"threads_started":1}
I20260807 18:06:26.108525 14743 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38685
I20260807 18:06:26.108901 14743 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:26.109647 14743 heartbeater.cc:511] Master 127.8.62.190:38685 requested a full tablet report, sending...
I20260807 18:06:26.111207 14280 ts_manager.cc:194] Registered new tserver with Master: 0142060f80d7453c9d783d405d62242d (127.8.62.131:41103)
I20260807 18:06:26.112112 14280 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:56123
I20260807 18:06:26.114147 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/test-admin.keytab.
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.142732 8442 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 18:06:26.148238 14751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:26.145690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.8.62.129:38603)
0807 18:06:26.146368 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.146375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.146401 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:06:26.146469 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.146475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.146699 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.146856 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.147824 (+ 968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58890: BlockingRecv error: recv got EOF from 127.0.0.1:58890 (error 108)
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":352,"threads_started":1}
W20260807 18:06:26.148689 8442 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.150291 8442 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.151893 8442 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:26.169111 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.153796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.8.62.129:38603)
0807 18:06:26.154051 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.154058 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.154286 (+ 228us) server_negotiation.cc:407] Connection header received
0807 18:06:26.154430 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.154433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.154497 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.154587 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.155670 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.156756 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.158035 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.158634 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.158739 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.162637 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.162653 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.162671 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.162716 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.165932 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.166827 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.166834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.166837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.166984 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.167376 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.167379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.167381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.167780 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.168010 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.168435 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.168817 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260807 18:06:26.176331 14751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:26.174033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.8.62.129:38603)
0807 18:06:26.174260 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.174271 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.174408 (+ 137us) server_negotiation.cc:407] Connection header received
0807 18:06:26.174555 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.174560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.174648 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.174749 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.176023 (+ 1274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58902: BlockingRecv error: recv got EOF from 127.0.0.1:58902 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 18:06:26.176524 8442 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.178319 8442 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.180027 8442 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:26.196623 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.181734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58914 (local address 127.8.62.129:38603)
0807 18:06:26.181973 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.181981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.182187 (+ 206us) server_negotiation.cc:407] Connection header received
0807 18:06:26.182429 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.182434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.182554 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.182652 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.183809 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.184723 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.186069 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.186518 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.186674 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.190879 (+ 4205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.190889 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.190893 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.190937 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.193868 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.194709 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.194714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.194716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.194851 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.195259 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.195262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.195264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.195539 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.195778 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.196016 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.196202 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 18:06:26.201601 14751 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:26.199600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58918 (local address 127.8.62.129:38603)
0807 18:06:26.199815 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.199822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.200003 (+ 181us) server_negotiation.cc:407] Connection header received
0807 18:06:26.200240 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.200246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.200322 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.200432 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.201283 (+ 851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58918: BlockingRecv error: recv got EOF from 127.0.0.1:58918 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:06:26.201713 8442 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786125932226923-8442-0/krb5kdc/krb5cc' of type FILE
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.203420 8442 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:26.204891 8442 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:26.219960 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.206297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58934 (local address 127.8.62.129:38603)
0807 18:06:26.206531 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.206538 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.206705 (+ 167us) server_negotiation.cc:407] Connection header received
0807 18:06:26.206831 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.206835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.206897 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.206981 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.207959 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.208788 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.209963 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.210379 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.210470 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.214401 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.214407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.214410 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.214503 (+ 93us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.217308 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.218021 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.218025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.218027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.218220 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.218554 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.218557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.218559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.218887 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.219186 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.219495 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.219691 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:06:26.221887 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14342
I20260807 18:06:26.234738 14751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.222713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.8.62.129:38603)
0807 18:06:26.222871 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.222875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.223248 (+ 373us) server_negotiation.cc:407] Connection header received
0807 18:06:26.223347 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.223350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.223412 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.223494 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.224620 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.225325 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.226626 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.227110 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.227210 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.229464 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.229468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.229471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.229504 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.232144 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.232840 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.232844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.232846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.232992 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.233398 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.233401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.233403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.233659 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.233870 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.234208 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.234410 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 18:06:26.235489 14466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:26.321309 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14342
I20260807 18:06:26.338975 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14477
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:06:26.355607 14764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.339924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35614 (local address 127.8.62.130:45023)
0807 18:06:26.340453 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.340460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.340485 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:06:26.340597 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.340608 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.340842 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.341031 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.342042 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.343078 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.344326 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.344972 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.345140 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.349252 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.349266 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.349281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.349321 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.352418 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.353405 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.353412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.353415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.353547 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.353959 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.353965 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.353968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.354345 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.354668 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.354988 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.355255 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
I20260807 18:06:26.357050 14602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:26.449571 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14477
I20260807 18:06:26.468065 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14613
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14235](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:06:26.486191 14766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:26.469329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55008 (local address 127.8.62.131:41103)
0807 18:06:26.470147 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:26.470155 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:26.470188 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:06:26.470252 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:26.470259 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:26.470549 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:26.470693 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:26.471781 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.473054 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.474294 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:26.475002 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:26.475129 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:26.479477 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:26.479491 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:26.479508 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:26.479548 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:26.482752 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.483540 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.483556 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.483558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.483705 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:26.484253 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:26.484258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:26.484261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:26.484898 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:26.485159 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:26.485500 (+ 341us) server_negotiation.cc:299] Negotiation successful
0807 18:06:26.485812 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":419,"threads_started":1}
I20260807 18:06:26.487617 14738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:26.572216 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14613
I20260807 18:06:26.590035 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14248
I20260807 18:06:26.591334 14311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:26.697506 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14248
2026-08-07T18:06:26Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4240 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14771](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14771](info): set up 2 sockets
Aug 07 18:06:26 dist-test-slave-rnrw krb5kdc[14771](info): commencing operation
krb5kdc: starting...
W20260807 18:06:27.828375 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:27 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:28Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab.
Entry for principal oryx/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab.
I20260807 18:06:28.064817 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44939
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab
--principal=oryx/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:28.416108 14784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:28.416445 14784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:28.416651 14784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:28.424293 14784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:28.424432 14784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:28.424525 14784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:28.424674 14784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:28.424731 14784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:28.435352 14784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab
--principal=oryx/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:28.436877 14784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:28.438288 14784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:28.451339 14789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:28.451295 14792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:28.451283 14790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:28.452247 14784 server_base.cc:1034] running on GCE node
I20260807 18:06:28.453217 14784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:28.454919 14784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:28.456223 14784 hybrid_clock.cc:648] HybridClock initialized: now 1786125988456177 us; error 38 us; skew 500 ppm
I20260807 18:06:28.456745 14784 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:28 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125988, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:28.460330 14784 init.cc:377] Logged in from keytab as oryx/127.8.62.190@KRBTEST.COM (short username oryx)
I20260807 18:06:28.462303 14784 webserver.cc:460] Webserver started at http://127.8.62.190:40445/ using document root <none> and password file <none>
I20260807 18:06:28.463073 14784 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:28.463248 14784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:28.463605 14784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:28.466921 14784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "9e0af4e45b0e4e4196365f0b3144e996"
format_stamp: "Formatted at 2026-08-07 18:06:28 on dist-test-slave-rnrw"
server_key: "c46808593963e19e5b97dd735272aeff"
server_key_iv: "e33544a9419bbbb15e2fedb96f193f1e"
server_key_version: "encryptionkey@0"
I20260807 18:06:28.467759 14784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "9e0af4e45b0e4e4196365f0b3144e996"
format_stamp: "Formatted at 2026-08-07 18:06:28 on dist-test-slave-rnrw"
server_key: "c46808593963e19e5b97dd735272aeff"
server_key_iv: "e33544a9419bbbb15e2fedb96f193f1e"
server_key_version: "encryptionkey@0"
I20260807 18:06:28.473649 14784 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:06:28.477608 14799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:28.479272 14784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:28.479516 14784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "9e0af4e45b0e4e4196365f0b3144e996"
format_stamp: "Formatted at 2026-08-07 18:06:28 on dist-test-slave-rnrw"
server_key: "c46808593963e19e5b97dd735272aeff"
server_key_iv: "e33544a9419bbbb15e2fedb96f193f1e"
server_key_version: "encryptionkey@0"
I20260807 18:06:28.479758 14784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:28.506283 14784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:28.510849 14784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:28.511173 14784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:28.531183 14784 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:44939
I20260807 18:06:28.531219 14853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:44939 every 8 connection(s)
I20260807 18:06:28.532743 14784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:28.536746 14854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:28.539592 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14784
I20260807 18:06:28.539819 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:28.540659 8442 external_mini_cluster.cc:1428] Setting key ee4222731349cbb471bdf759785884d5
I20260807 18:06:28.549957 14854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Bootstrap starting.
I20260807 18:06:28.553333 14854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:28.554414 14854 log.cc:824] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:06:28 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:28.557912 14854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: No bootstrap required, opened a new log
I20260807 18:06:28.564945 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:28.543020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44578 (local address 127.8.62.190:44939)
0807 18:06:28.544085 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:28.544099 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:28.544145 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:06:28.545965 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:28.545990 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:28.546413 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:28.546892 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:28.548585 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:28.549958 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:28.551362 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:28.552109 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:28.552325 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:28.556966 (+ 4641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:28.556975 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:28.556991 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:28.557028 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:28.560409 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:28.561336 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:28.561343 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:28.561350 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:28.561532 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:28.561966 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:28.561969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:28.561971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:28.562525 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:28.562809 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:28.563253 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 18:06:28.563717 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":460,"threads_started":1}
I20260807 18:06:28.566962 14854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } }
I20260807 18:06:28.567366 14854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:28.567512 14854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e0af4e45b0e4e4196365f0b3144e996, State: Initialized, Role: FOLLOWER
I20260807 18:06:28.568187 14854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } }
I20260807 18:06:28.568435 14854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:28.568588 14854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:28.568794 14854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:28.571055 14854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } }
I20260807 18:06:28.571609 14854 leader_election.cc:302] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e0af4e45b0e4e4196365f0b3144e996; no voters:
I20260807 18:06:28.572366 14854 leader_election.cc:288] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:28.572566 14859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:28.574131 14859 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 LEADER]: Becoming Leader. State: Replica: 9e0af4e45b0e4e4196365f0b3144e996, State: Running, Role: LEADER
I20260807 18:06:28.574673 14859 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } }
I20260807 18:06:28.574887 14854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:28.577659 14861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0af4e45b0e4e4196365f0b3144e996. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } }
I20260807 18:06:28.578152 14861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:28.578104 14860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } }
I20260807 18:06:28.578485 14860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:28.580132 14868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:28.585664 14868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:28.594966 14868 catalog_manager.cc:1347] Generated new cluster ID: b09efc591e7749d8a2c7f618beea1679
I20260807 18:06:28.595109 14868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:28.607544 14868 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:28.609021 14868 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:28.618299 14868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Generated new TSK 0
I20260807 18:06:28.619328 14868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
I20260807 18:06:28.677093 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:29.036298 14880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:29.036708 14880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:29.036923 14880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:29.044457 14880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:29.044688 14880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:29.044900 14880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:29.055303 14880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:29.056952 14880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:29.058511 14880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:29.073135 14885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:29.073032 14886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:29.072999 14888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:29.074230 14880 server_base.cc:1034] running on GCE node
I20260807 18:06:29.075095 14880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:29.076314 14880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:29.077652 14880 hybrid_clock.cc:648] HybridClock initialized: now 1786125989077574 us; error 72 us; skew 500 ppm
I20260807 18:06:29.078110 14880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:29 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125989, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:29.082050 14880 init.cc:377] Logged in from keytab as oryx/127.8.62.129@KRBTEST.COM (short username oryx)
I20260807 18:06:29.083752 14880 webserver.cc:460] Webserver started at http://127.8.62.129:34355/ using document root <none> and password file <none>
I20260807 18:06:29.084532 14880 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:29.084723 14880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:29.085062 14880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:29.088227 14880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "58d35ba8e6f5c7f84b85a314346056f8"
server_key_iv: "00e85a24b5803b9f6d4c56e8bfceb766"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.089143 14880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "58d35ba8e6f5c7f84b85a314346056f8"
server_key_iv: "00e85a24b5803b9f6d4c56e8bfceb766"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.095183 14880 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:06:29.099291 14896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.101089 14880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:06:29.101306 14880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "58d35ba8e6f5c7f84b85a314346056f8"
server_key_iv: "00e85a24b5803b9f6d4c56e8bfceb766"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.101541 14880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:29.118636 14880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:29.123802 14880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:29.124195 14880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:29.125492 14880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:29.127019 14880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:29.127133 14880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.127285 14880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:29.127369 14880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.163859 14880 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:41749
I20260807 18:06:29.163908 15011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:41749 every 8 connection(s)
I20260807 18:06:29.165444 14880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:29.173121 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 14880
I20260807 18:06:29.173369 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:29.174074 8442 external_mini_cluster.cc:1428] Setting key 72f97182ccdfedd261af893e1e4a7cd2
Aug 07 18:06:29 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125989, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:29.192106 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:29.169080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:33497 (local address 127.8.62.190:44939)
0807 18:06:29.169676 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:29.169681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:29.170604 (+ 923us) server_negotiation.cc:407] Connection header received
0807 18:06:29.172645 (+ 2041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:29.172656 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:29.172753 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:29.172894 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:29.174951 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.176028 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.178081 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.178947 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.179125 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:29.184403 (+ 5278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:29.184410 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:29.184413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:29.184461 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:29.187648 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:29.188990 (+ 1342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:29.188996 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:29.188998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:29.189182 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:29.189881 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:29.189886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:29.189888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:29.190231 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:29.190368 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:29.191576 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0807 18:06:29.191809 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":281,"threads_started":1}
I20260807 18:06:29.193862 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:29.169505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:33497)
0807 18:06:29.170347 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:29.170400 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:29.172242 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:29.173183 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:29.173194 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:29.173894 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:29.174694 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:29.174718 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.176257 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.176263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:29.177875 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:29.177892 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.179247 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.179261 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:29.179505 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:29.180491 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:29.180822 (+ 331us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:29.184069 (+ 3247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:29.187899 (+ 3830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:29.187913 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:29.187937 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:29.188805 (+ 868us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:29.189350 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:29.189358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:29.189365 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:29.189700 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:29.190390 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:29.190401 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:29.191004 (+ 603us) client_negotiation.cc:769] Sending connection context
0807 18:06:29.191388 (+ 384us) client_negotiation.cc:240] Negotiation successful
0807 18:06:29.192070 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":320,"threads_started":1}
I20260807 18:06:29.195957 15012 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:29.196341 15012 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:29.197253 15012 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:29.199738 14816 ts_manager.cc:194] Registered new tserver with Master: ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:29.201571 14816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:33497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
I20260807 18:06:29.233150 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:29.584828 15019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:29.585132 15019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:29.585328 15019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:29.592749 15019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:29.592881 15019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:29.593084 15019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:29.603430 15019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:29.604988 15019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:29.606616 15019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:29.622324 15027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:29.622267 15025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:29.622370 15024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:29.623184 15019 server_base.cc:1034] running on GCE node
I20260807 18:06:29.623889 15019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:29.625049 15019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:29.626345 15019 hybrid_clock.cc:648] HybridClock initialized: now 1786125989626289 us; error 50 us; skew 500 ppm
I20260807 18:06:29.626971 15019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:29 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125989, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:29.631040 15019 init.cc:377] Logged in from keytab as oryx/127.8.62.130@KRBTEST.COM (short username oryx)
I20260807 18:06:29.632938 15019 webserver.cc:460] Webserver started at http://127.8.62.130:41907/ using document root <none> and password file <none>
I20260807 18:06:29.633723 15019 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:29.633831 15019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:29.634093 15019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:29.637472 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "055a3f0da5a74e959525874718da97bd"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "e5b836458b5fbac5132a01a5484f5cc8"
server_key_iv: "678d2873aac713d7fb189e327ca7be87"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.638271 15019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "055a3f0da5a74e959525874718da97bd"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "e5b836458b5fbac5132a01a5484f5cc8"
server_key_iv: "678d2873aac713d7fb189e327ca7be87"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.644136 15019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:29.648684 15034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.650533 15019 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:06:29.650719 15019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "055a3f0da5a74e959525874718da97bd"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "e5b836458b5fbac5132a01a5484f5cc8"
server_key_iv: "678d2873aac713d7fb189e327ca7be87"
server_key_version: "encryptionkey@0"
I20260807 18:06:29.650947 15019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:29.678454 15019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:29.683678 15019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:29.683993 15019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:29.685240 15019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:29.686764 15019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:29.686862 15019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.686962 15019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:29.687009 15019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:29.725834 15019 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:39541
I20260807 18:06:29.725885 15147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:39541 every 8 connection(s)
I20260807 18:06:29.727546 15019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:29.737886 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15019
I20260807 18:06:29.738076 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:29.738550 8442 external_mini_cluster.cc:1428] Setting key cf921c6fa17590ef39002b8f626576e2
Aug 07 18:06:29 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125989, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:29.749665 15150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:29.730520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:39225 (local address 127.8.62.190:44939)
0807 18:06:29.731009 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:29.731013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:29.732151 (+ 1138us) server_negotiation.cc:407] Connection header received
0807 18:06:29.734193 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:29.734200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:29.734317 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:29.734448 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:29.736089 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.736928 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.738661 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.739109 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.739265 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:29.743116 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:29.743121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:29.743124 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:29.743157 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:29.745940 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:29.746768 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:29.746776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:29.746784 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:29.746917 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:29.747392 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:29.747398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:29.747401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:29.747867 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:29.748042 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:29.748883 (+ 841us) server_negotiation.cc:299] Negotiation successful
0807 18:06:29.749159 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":246,"threads_started":1}
I20260807 18:06:29.750423 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:29.731178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:39225)
0807 18:06:29.731969 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:29.732015 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:29.733873 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:29.734704 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:29.734713 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:29.735274 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:29.735888 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:29.735906 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.737105 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.737109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:29.738511 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:29.738524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:29.739364 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:29.739374 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:29.739638 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:29.740435 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:29.740474 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:29.742854 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:29.746115 (+ 3261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:29.746125 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:29.746145 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:29.746621 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:29.747045 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:29.747050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:29.747054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:29.747269 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:29.748024 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:29.748030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:29.748422 (+ 392us) client_negotiation.cc:769] Sending connection context
0807 18:06:29.748759 (+ 337us) client_negotiation.cc:240] Negotiation successful
0807 18:06:29.749203 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":340,"threads_started":1}
I20260807 18:06:29.752110 15148 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:29.752477 15148 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:29.753476 15148 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
I20260807 18:06:29.755218 14816 ts_manager.cc:194] Registered new tserver with Master: 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541)
I20260807 18:06:29.756103 14816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:39225
WARNING: no policy specified for oryx/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
I20260807 18:06:29.796856 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:30.138048 15154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:30.138389 15154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:30.138566 15154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:30.145959 15154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:30.146152 15154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:30.146354 15154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:30.157111 15154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:44939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:30.158684 15154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:30.160219 15154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:30.175385 15159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:30.176222 15160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:30.177726 15162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:30.178041 15154 server_base.cc:1034] running on GCE node
I20260807 18:06:30.178793 15154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:30.179972 15154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:30.181245 15154 hybrid_clock.cc:648] HybridClock initialized: now 1786125990181191 us; error 54 us; skew 500 ppm
I20260807 18:06:30.181658 15154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125990, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:30.185462 15154 init.cc:377] Logged in from keytab as oryx/127.8.62.131@KRBTEST.COM (short username oryx)
I20260807 18:06:30.187178 15154 webserver.cc:460] Webserver started at http://127.8.62.131:42357/ using document root <none> and password file <none>
I20260807 18:06:30.187979 15154 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:30.188125 15154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:30.188453 15154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:30.191707 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "5948fd37fba0439fb7ff451085ef9d49"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "9cb59ded353eb7b6b1caca10cdd7cec2"
server_key_iv: "5305ccfd234ca39a34226aa3a9555ba7"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.192628 15154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "5948fd37fba0439fb7ff451085ef9d49"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "9cb59ded353eb7b6b1caca10cdd7cec2"
server_key_iv: "5305ccfd234ca39a34226aa3a9555ba7"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.198462 15154 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:06:30.202529 15169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:30.204082 15154 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:06:30.204283 15154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "5948fd37fba0439fb7ff451085ef9d49"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "9cb59ded353eb7b6b1caca10cdd7cec2"
server_key_iv: "5305ccfd234ca39a34226aa3a9555ba7"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.204491 15154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:30.205156 15012 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
I20260807 18:06:30.222167 15154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:30.226509 15154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:30.226832 15154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:30.227887 15154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:30.229463 15154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:30.229539 15154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:30.229719 15154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:30.229779 15154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:30.265275 15154 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:33865
I20260807 18:06:30.265323 15283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:33865 every 8 connection(s)
I20260807 18:06:30.266728 15154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:30.270728 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15154
I20260807 18:06:30.270955 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:30.271500 8442 external_mini_cluster.cc:1428] Setting key b69fb7c71f149d9c9be0e03ae7fde4e8
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125990, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:30.290911 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.269716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:35111 (local address 127.8.62.190:44939)
0807 18:06:30.270235 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.270241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.271687 (+ 1446us) server_negotiation.cc:407] Connection header received
0807 18:06:30.274064 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.274070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.274192 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.274335 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.276284 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.277155 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.278935 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.279452 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.279554 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.283531 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.283536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.283540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.283588 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.286945 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.288135 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.288146 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.288148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.288333 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.288908 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.288914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.288917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.289234 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.289422 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.290242 (+ 820us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.290501 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260807 18:06:30.291944 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.270318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:35111)
0807 18:06:30.271358 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.271417 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.273753 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.274636 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.274681 (+ 45us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.275379 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.276057 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.276086 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.277317 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.277321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.278784 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.278797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.279698 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.279704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.279933 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.280727 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.280752 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.283244 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.287139 (+ 3895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.287149 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.287167 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.287966 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.288471 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.288476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.288480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.288779 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.289379 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.289388 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.289827 (+ 439us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.290128 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.290737 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":510,"threads_started":1}
I20260807 18:06:30.293721 15284 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:30.294123 15284 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:30.294975 15284 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
I20260807 18:06:30.296828 14816 ts_manager.cc:194] Registered new tserver with Master: 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865)
I20260807 18:06:30.297713 14816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:35111
I20260807 18:06:30.301194 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.189.keytab.
Entry for principal oryx/127.8.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.189.keytab.
I20260807 18:06:30.357960 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:37593
--webserver_interface=127.8.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.189.keytab
--principal=oryx/127.8.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44939,127.8.62.189:37593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:30.715430 15290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:30.715930 15290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:30.716089 15290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:30.724085 15290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:30.724263 15290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:30.724354 15290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:30.724443 15290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:30.724511 15290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:30.735514 15290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44939,127.8.62.189:37593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.189.keytab
--principal=oryx/127.8.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:37593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
W20260807 18:06:30.735975 15290 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.62.190:44939,127.8.62.189:37593'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 18:06:30.737166 15290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:30.738534 15290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:30.751178 15295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:30.751292 15296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:30.751782 15298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:30.752666 15290 server_base.cc:1034] running on GCE node
I20260807 18:06:30.753600 15290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:30.755306 15290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:30.756635 15290 hybrid_clock.cc:648] HybridClock initialized: now 1786125990756472 us; error 41 us; skew 500 ppm
I20260807 18:06:30.757032 15290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:06:30.759071 15148 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125990, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:30.760951 15290 init.cc:377] Logged in from keytab as oryx/127.8.62.189@KRBTEST.COM (short username oryx)
I20260807 18:06:30.762743 15290 webserver.cc:460] Webserver started at http://127.8.62.189:35915/ using document root <none> and password file <none>
I20260807 18:06:30.763558 15290 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:30.763652 15290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:30.764010 15290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:30.767263 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/instance:
uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "7c361cc3f5c7bfcdf5c0431850575678"
server_key_iv: "6a63988496cf313ec86168153f3685ee"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.768193 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal/instance:
uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "7c361cc3f5c7bfcdf5c0431850575678"
server_key_iv: "6a63988496cf313ec86168153f3685ee"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.774083 15290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:06:30.779153 15305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:30.781319 15290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:06:30.781543 15290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "7c361cc3f5c7bfcdf5c0431850575678"
server_key_iv: "6a63988496cf313ec86168153f3685ee"
server_key_version: "encryptionkey@0"
I20260807 18:06:30.781837 15290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:30.798066 15290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:30.802866 15290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:30.803172 15290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:30.823415 15290 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.189:37593
I20260807 18:06:30.823424 15357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.189:37593 every 8 connection(s)
I20260807 18:06:30.824988 15290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
I20260807 18:06:30.829164 15358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:30.833154 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15290
I20260807 18:06:30.836666 15358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125990, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.189@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:30.858387 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.841014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.8.62.190:44939)
0807 18:06:30.841580 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.841587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.842074 (+ 487us) server_negotiation.cc:407] Connection header received
0807 18:06:30.842313 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.842317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.842393 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.842519 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.843816 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.844737 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.846506 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.846926 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.847029 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.851422 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.851428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.851431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.851516 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.854805 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.855699 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.855704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.855707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.855834 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.856275 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.856285 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.856287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.856818 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.856973 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.857761 (+ 788us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.857999 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":259,"threads_started":1}
I20260807 18:06:30.859020 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.835585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39906 (local address 127.8.62.189:37593)
0807 18:06:30.836440 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.836464 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.836516 (+ 52us) server_negotiation.cc:407] Connection header received
0807 18:06:30.838780 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.838812 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.839377 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.839825 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.841722 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.843301 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.844734 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.845503 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.845640 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.850866 (+ 5226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.850881 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.850899 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.850940 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.853867 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.854784 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.854791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.854799 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.855045 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.855535 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.855541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.855543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.856383 (+ 840us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.856676 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.857225 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.857726 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":367,"threads_started":1}
I20260807 18:06:30.859206 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.840995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44584)
0807 18:06:30.841931 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.841970 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.842162 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.842586 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.842595 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.843259 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.843632 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.843644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.844894 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.844897 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.846318 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.846331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.847184 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.847193 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.847407 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.848105 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.848141 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.851211 (+ 3070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.854935 (+ 3724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.854945 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.854951 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.855535 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.855953 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.855963 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.855966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.856101 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.856952 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.856962 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.857225 (+ 263us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.857435 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.857860 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":132,"spinlock_wait_cycles":22784,"thread_start_us":543,"threads_started":1}
I20260807 18:06:30.861229 15358 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125990, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.189@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
I20260807 18:06:30.874967 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.862042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39908)
0807 18:06:30.862529 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.862550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.862665 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.862962 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.862965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.863386 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.863624 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.863635 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.864682 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.864686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.865618 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.865631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.866290 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.866294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.866432 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.867078 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.867096 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.869663 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.872658 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.872662 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.872665 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.873043 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.873391 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.873394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.873396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.873532 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.874148 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.874151 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.874273 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.874378 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.874617 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350}
I20260807 18:06:30.875047 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.862307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.8.62.189:37593)
0807 18:06:30.862489 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.862494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.862640 (+ 146us) server_negotiation.cc:407] Connection header received
0807 18:06:30.862760 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.862763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.862820 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.862898 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.863757 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.864449 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.865755 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.866146 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.866245 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.869861 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.869866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.869869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.869901 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.872455 (+ 2554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.873161 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.873165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.873167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.873289 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.873684 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.873689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.873691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.874032 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.874157 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.874525 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.874706 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:06:30.880333 15358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Bootstrap starting.
I20260807 18:06:30.883700 15358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:30.885010 15358 log.cc:824] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:30.887959 15358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: No bootstrap required, opened a new log
I20260807 18:06:30.895430 15358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } }
I20260807 18:06:30.895800 15358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:30.895869 15358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea4e7b4593a4c3d80fe664c2fffb6fb, State: Initialized, Role: FOLLOWER
I20260807 18:06:30.896435 15358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } }
I20260807 18:06:30.897413 15365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } } }
I20260807 18:06:30.897800 15365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:30.898526 15358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:06:30.908644 15376 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:06:30.908810 15376 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:06:30.912122 15290 master_runner.cc:428] Detected that this master 5ea4e7b4593a4c3d80fe664c2fffb6fb is joining an existing cluster
I20260807 18:06:30.912240 15290 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.62.189:37593
I20260807 18:06:30.936519 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.917237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44594 (local address 127.8.62.190:44939)
0807 18:06:30.917453 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.917460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.918091 (+ 631us) server_negotiation.cc:407] Connection header received
0807 18:06:30.918382 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.918390 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.918531 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.918613 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.920626 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.922151 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.925021 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.925971 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.926104 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.929189 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.929195 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.929204 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.929241 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.931923 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.932897 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.932904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.932907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.933086 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.933640 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.933645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.933649 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.934049 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.934197 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.935895 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.936156 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:06:30.937738 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.917264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44594)
0807 18:06:30.917989 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.918010 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.918205 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.918862 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.918870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.919811 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.920339 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.920357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.922419 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.922431 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.924787 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.924809 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.926537 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.926547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.926804 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.927702 (+ 898us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.927735 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.928968 (+ 1233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.932084 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.932092 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.932102 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.932754 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.933247 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.933251 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.933260 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.933494 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.934276 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.934280 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.935541 (+ 1261us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.936839 (+ 1298us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.937196 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"mutex_wait_us":4,"thread_start_us":402,"threads_started":1}
WARNING: no policy specified for oryx/127.8.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.188@KRBTEST.COM" created.
I20260807 18:06:30.948787 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.918611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39922 (local address 127.8.62.189:37593)
0807 18:06:30.918801 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.918807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.932420 (+ 13613us) server_negotiation.cc:407] Connection header received
0807 18:06:30.932678 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.932685 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.932752 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.932824 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.934276 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.935044 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.937245 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.938037 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.938109 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.940766 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.940772 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.940776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.940812 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.944339 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.946148 (+ 1809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.946154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.946159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.946368 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.947017 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.947024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.947030 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.947406 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.947576 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.948125 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.948358 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":58}
I20260807 18:06:30.949905 15383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.918175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39922)
0807 18:06:30.932152 (+ 13977us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.932174 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.932519 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.933061 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.933066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.933781 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.934140 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.934152 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.935230 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.935235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.937094 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.937108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.938542 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.938548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.938726 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.939386 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.939407 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.940584 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.944497 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.944501 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.944505 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.945960 (+ 1455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.946551 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.946558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.946563 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.946823 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.947642 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.947647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.947789 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.949135 (+ 1346us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.949428 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12503,"mutex_wait_us":1,"thread_start_us":9778,"threads_started":1}
Aug 07 18:06:30 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125988, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.190@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
I20260807 18:06:30.975353 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.945943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.8.62.189:37593)
0807 18:06:30.949100 (+ 3157us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:30.949106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:30.955685 (+ 6579us) server_negotiation.cc:407] Connection header received
0807 18:06:30.956140 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:30.956148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:30.956268 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:30.956395 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:30.958005 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.959307 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.961391 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.962568 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.962731 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.967564 (+ 4833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:30.967574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:30.967581 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:30.967637 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:30.971741 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.972879 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.972884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.972886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.973045 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:30.973558 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:30.973562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:30.973564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:30.973867 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:30.973984 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:30.974682 (+ 698us) server_negotiation.cc:299] Negotiation successful
0807 18:06:30.975003 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3054,"thread_start_us":8596,"threads_started":1}
I20260807 18:06:30.975396 15385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:30.945572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39928)
0807 18:06:30.955282 (+ 9710us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:30.955332 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:30.955862 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:30.956464 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:30.956472 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:30.957468 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:30.957793 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.957807 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.959471 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.959477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.961222 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:30.961239 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:30.962774 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:30.962781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:30.963023 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:30.963960 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:30.963986 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:30.967246 (+ 3260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:30.971921 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.971941 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.971958 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.972676 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.973235 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:30.973241 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:30.973245 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:30.973430 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:30.974020 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:30.974030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:30.974271 (+ 241us) client_negotiation.cc:769] Sending connection context
0807 18:06:30.974491 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 18:06:30.974876 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9393,"thread_start_us":9248,"threads_started":1}
I20260807 18:06:30.978092 14816 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.62.189:37593: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
member_type: NON_VOTER
last_known_addr {
host: "127.8.62.189"
port: 37593
}
attrs {
promote: true
}
}
dest_uuid: "9e0af4e45b0e4e4196365f0b3144e996"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.188.keytab.
Entry for principal oryx/127.8.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.188.keytab.
I20260807 18:06:30.979341 14816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } }
I20260807 18:06:30.981084 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:38419
--webserver_interface=127.8.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.188.keytab
--principal=oryx/127.8.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:06:30.993198 15333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:30.993070 15390 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } }
I20260807 18:06:30.997445 15333 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 18:06:30.998847 15392 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:30.998588 15390 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.62.189:37593
I20260807 18:06:31.000527 15290 master.cc:537] Master@127.8.62.189:37593 shutting down...
W20260807 18:06:31.003647 14801 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 18:06:31.016044 14801 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 -> Peer 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189:37593): Couldn't send request to peer 5ea4e7b4593a4c3d80fe664c2fffb6fb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:06:31.016856 15389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } }
I20260807 18:06:31.017288 15389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:31.018186 15290 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:31.018671 15290 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:31.018873 15290 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: stopping tablet replica
I20260807 18:06:31.026953 15290 master.cc:552] Master@127.8.62.189:37593 shutdown complete.
I20260807 18:06:31.027072 15290 master_runner.cc:305] Clearing existing system tablet
I20260807 18:06:31.027635 15290 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:31.027760 15290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:31.031610 15290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:06:31.033624 15398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:31.034013 15290 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:31.034291 15290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:06:31.034399 15290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "7c361cc3f5c7bfcdf5c0431850575678"
server_key_iv: "6a63988496cf313ec86168153f3685ee"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.040140 15290 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:31.048734 15290 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:06:31.048985 15290 log.cc:1196] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 18:06:31.049546 15290 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Deleting consensus metadata
I20260807 18:06:31.049886 15290 master_runner.cc:315] Copying system tablet from 127.8.62.190:44939
I20260807 18:06:31.052937 15290 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: tablet copy: Beginning tablet copy session from remote peer at address 127.8.62.190:44939
I20260807 18:06:31.077440 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.053851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44596 (local address 127.8.62.190:44939)
0807 18:06:31.054062 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.054072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.056363 (+ 2291us) server_negotiation.cc:407] Connection header received
0807 18:06:31.056612 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.056618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.056761 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.056893 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.058210 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.059398 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.061641 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.062617 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.062808 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.069586 (+ 6778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.069594 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.069607 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.069657 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.073752 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.074813 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.074820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.074823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.075066 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.075607 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.075612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.075615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.076079 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.076250 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.076799 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.077059 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:06:31.077571 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.053778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44596)
0807 18:06:31.056184 (+ 2406us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.056215 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.056418 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.056952 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.056958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.057638 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.057999 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.058015 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.059592 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.059599 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.061410 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.061432 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.062864 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.062872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.063098 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.067859 (+ 4761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.067890 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.069230 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.073986 (+ 4756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.073994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.073998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.074624 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.075218 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.075224 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.075228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.075466 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.076229 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.076235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.076440 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.076678 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.077071 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2255,"thread_start_us":2069,"threads_started":1}
I20260807 18:06:31.078769 14837 tablet_copy_service.cc:140] P 9e0af4e45b0e4e4196365f0b3144e996: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5ea4e7b4593a4c3d80fe664c2fffb6fb ({username='oryx', principal='oryx/127.8.62.189@KRBTEST.COM'} at 127.0.0.1:44596)
I20260807 18:06:31.079066 14837 tablet_copy_service.cc:161] P 9e0af4e45b0e4e4196365f0b3144e996: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5ea4e7b4593a4c3d80fe664c2fffb6fb at {username='oryx', principal='oryx/127.8.62.189@KRBTEST.COM'} at 127.0.0.1:44596: session id = 5ea4e7b4593a4c3d80fe664c2fffb6fb-00000000000000000000000000000000
I20260807 18:06:31.083528 14837 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:06:31.086272 15290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:06:31.094661 15290 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: tablet copy: Starting download of 0 data blocks...
I20260807 18:06:31.095077 15290 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: tablet copy: Starting download of 1 WAL segments...
I20260807 18:06:31.098791 15290 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:06:31.104362 14837 tablet_copy_service.cc:342] P 9e0af4e45b0e4e4196365f0b3144e996: Request end of tablet copy session 5ea4e7b4593a4c3d80fe664c2fffb6fb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.62.189@KRBTEST.COM'} at 127.0.0.1:44596
I20260807 18:06:31.104681 14837 tablet_copy_service.cc:433] P 9e0af4e45b0e4e4196365f0b3144e996: ending tablet copy session 5ea4e7b4593a4c3d80fe664c2fffb6fb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5ea4e7b4593a4c3d80fe664c2fffb6fb
I20260807 18:06:31.117733 15290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:31.120769 15406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:31.125003 15407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:31.131076 15290 server_base.cc:1034] running on GCE node
W20260807 18:06:31.131980 15409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:31.132647 15290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:31.133492 15290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:31.134853 15290 hybrid_clock.cc:648] HybridClock initialized: now 1786125991134752 us; error 46 us; skew 500 ppm
I20260807 18:06:31.135196 15290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125991, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:31.137831 15290 init.cc:377] Logged in from keytab as oryx/127.8.62.189@KRBTEST.COM (short username oryx)
I20260807 18:06:31.141690 15290 webserver.cc:460] Webserver started at http://127.8.62.189:43597/ using document root <none> and password file <none>
I20260807 18:06:31.142256 15290 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:31.142477 15290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:31.146569 15290 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:06:31.149029 15417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:31.149549 15290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260807 18:06:31.149773 15290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "7c361cc3f5c7bfcdf5c0431850575678"
server_key_iv: "6a63988496cf313ec86168153f3685ee"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.150031 15290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:31.171013 15290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:31.175843 15290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:31.217401 15290 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.189:37593
I20260807 18:06:31.217478 15469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.189:37593 every 8 connection(s)
I20260807 18:06:31.218550 15290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
I20260807 18:06:31.224144 15470 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:06:31.228936 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Bootstrap starting.
I20260807 18:06:31.251041 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:06:31.251822 15470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: Bootstrap complete.
I20260807 18:06:31.253077 15470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } }
I20260807 18:06:31.253376 15470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5ea4e7b4593a4c3d80fe664c2fffb6fb, State: Initialized, Role: LEARNER
I20260807 18:06:31.253659 15470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } }
I20260807 18:06:31.254598 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } }
I20260807 18:06:31.254990 15470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:31.255003 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: LEARNER
I20260807 18:06:31.265695 15485 catalog_manager.cc:1259] Loaded cluster ID: b09efc591e7749d8a2c7f618beea1679
I20260807 18:06:31.265985 15485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: loading cluster ID for follower catalog manager: success
I20260807 18:06:31.269886 15485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: acquiring CA information for follower catalog manager: success
I20260807 18:06:31.272435 15485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:06:31.301405 15284 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
W20260807 18:06:31.559849 15391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:31.560225 15391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:31.560433 15391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:31.568495 15391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:31.568724 15391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:31.568791 15391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:31.568871 15391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:31.568925 15391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:31.581245 15391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.188.keytab
--principal=oryx/127.8.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:31.582779 15391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:31.584292 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:31.597504 15491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:31.597414 15490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:31.597455 15493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:31.602346 15391 server_base.cc:1034] running on GCE node
I20260807 18:06:31.603241 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:31.604909 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:31.606142 15391 hybrid_clock.cc:648] HybridClock initialized: now 1786125991606095 us; error 49 us; skew 500 ppm
I20260807 18:06:31.606597 15391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125991, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:31.610347 15391 init.cc:377] Logged in from keytab as oryx/127.8.62.188@KRBTEST.COM (short username oryx)
I20260807 18:06:31.612243 15391 webserver.cc:460] Webserver started at http://127.8.62.188:36159/ using document root <none> and password file <none>
I20260807 18:06:31.613075 15391 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:31.613220 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:31.613551 15391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:31.616928 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/instance:
uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
format_stamp: "Formatted at 2026-08-07 18:06:31 on dist-test-slave-rnrw"
server_key: "135b9785827c1d08ab2f3a325eb9225d"
server_key_iv: "996faf03c6b8d45b6123cea47afd64bc"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.617921 15391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal/instance:
uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
format_stamp: "Formatted at 2026-08-07 18:06:31 on dist-test-slave-rnrw"
server_key: "135b9785827c1d08ab2f3a325eb9225d"
server_key_iv: "996faf03c6b8d45b6123cea47afd64bc"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.623807 15391 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:31.628726 15503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:31.631176 15391 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:31.631455 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
format_stamp: "Formatted at 2026-08-07 18:06:31 on dist-test-slave-rnrw"
server_key: "135b9785827c1d08ab2f3a325eb9225d"
server_key_iv: "996faf03c6b8d45b6123cea47afd64bc"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.631736 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:31.632017 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.624574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39932)
0807 18:06:31.625100 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.625127 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.625307 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.626086 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.626090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.626117 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:31.626335 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.626344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.628029 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.628034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.629473 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.629497 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.630938 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.630944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.631176 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.631283 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.631402 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.631573 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":238,"threads_started":1}
I20260807 18:06:31.632141 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.624919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.8.62.189:37593)
0807 18:06:31.625623 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.625629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.625658 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:06:31.625745 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.625751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.625855 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.625995 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:31.626585 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.627847 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.629691 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.630694 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.630889 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.631274 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.631464 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.631649 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":335,"threads_started":1}
I20260807 18:06:31.633236 15444 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 18:06:31.633816 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0af4e45b0e4e4196365f0b3144e996. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } }
I20260807 18:06:31.634091 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: LEARNER
I20260807 18:06:31.661000 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:31.667366 15391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:31.667662 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:31.698918 15391 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.188:38419
I20260807 18:06:31.698976 15557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.188:38419 every 8 connection(s)
I20260807 18:06:31.700510 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
I20260807 18:06:31.705302 15558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:31.708571 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15391
I20260807 18:06:31.714437 15558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125991, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.188@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:31.738386 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.711761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.8.62.188:38419)
0807 18:06:31.713229 (+ 1468us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.713242 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.713298 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:06:31.715022 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.715048 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.715493 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.715863 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.718139 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.720683 (+ 2544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.722629 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.723422 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.723578 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.728974 (+ 5396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.728988 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.729008 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.729070 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.733111 (+ 4041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.733966 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.733980 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.733988 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.734160 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.734690 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.734695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.734699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.735315 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.735745 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.736353 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.736955 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1237,"thread_start_us":924,"threads_started":1}
I20260807 18:06:31.739673 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.718472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44604)
0807 18:06:31.720727 (+ 2255us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.720823 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.721070 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.723324 (+ 2254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.723339 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.724357 (+ 1018us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.724768 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.724786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.726420 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.726426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.727791 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.727805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.728701 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.728709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.728962 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.729705 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.729726 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.732818 (+ 3092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.736573 (+ 3755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.736588 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.736598 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.737272 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.737735 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.737741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.737745 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.737923 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.738727 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.738737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.738916 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.739089 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.739348 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1816,"thread_start_us":1561,"threads_started":1}
I20260807 18:06:31.740031 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.718585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.8.62.190:44939)
0807 18:06:31.722944 (+ 4359us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.722949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.722969 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:06:31.723022 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.723026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.723088 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.723265 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.724981 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.726259 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.727924 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.728397 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.728578 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.733120 (+ 4542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.733127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.733130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.733171 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.736287 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.737422 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.737428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.737431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.737615 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.738148 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.738152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.738154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.738469 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.738646 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.739254 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.739499 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4216,"thread_start_us":4108,"threads_started":1}
I20260807 18:06:31.742256 15558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125991, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.188@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
I20260807 18:06:31.761694 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.742922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39936)
0807 18:06:31.743163 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.743180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.743265 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.743809 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.743813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.744142 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.744285 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.744293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.745576 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.745580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.746573 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.746585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.747721 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.747726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.747879 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.749123 (+ 1244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.749142 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.752915 (+ 3773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.758388 (+ 5473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.758395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.758399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.759021 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.759554 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.759561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.759566 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.759800 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.760742 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.760749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.760930 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.761116 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.761391 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:06:31.761981 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.743339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.8.62.189:37593)
0807 18:06:31.743542 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.743546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.743562 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:06:31.743612 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.743615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.743685 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.743763 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.744439 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.745394 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.746728 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.747547 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.747650 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.753210 (+ 5560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.753218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.753223 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.753289 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.758154 (+ 4865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.759214 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.759225 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.759231 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.759426 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.759956 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.759963 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.759968 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.760432 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.760667 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.761251 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.761600 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:06:31.763686 15558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125991, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.188@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
I20260807 18:06:31.781934 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.764821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.8.62.188:38419)
0807 18:06:31.765269 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.765274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.765308 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:06:31.765354 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.765359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.765449 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.765560 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.766575 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.767840 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.769734 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.770286 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.770432 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.774504 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.774514 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.774518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.774546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.778837 (+ 4291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.779751 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.779755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.779757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.779863 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.780322 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.780326 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.780329 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.780741 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.780898 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.781257 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.781612 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":132,"server-negotiator.queue_time_us":317}
I20260807 18:06:31.781949 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.764620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.0.0.1:46744)
0807 18:06:31.764867 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.764887 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.765025 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.765613 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.765618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.766172 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.766417 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.766429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.767985 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.767993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.769570 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.769612 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.770451 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.770456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.770695 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.771396 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.771423 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.774288 (+ 2865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.778979 (+ 4691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.778986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.778990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.779592 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.779998 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.780002 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.780011 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.780202 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.780858 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.780862 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.781017 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.781161 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.781464 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:06:31.787571 15558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Bootstrap starting.
I20260807 18:06:31.792150 15558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:31.793632 15558 log.cc:824] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:31.797060 15558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: No bootstrap required, opened a new log
I20260807 18:06:31.808969 15558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } }
I20260807 18:06:31.809382 15558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:31.809545 15558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d8e31cd37f34c1b89577c57f1c3e014, State: Initialized, Role: FOLLOWER
I20260807 18:06:31.810264 15558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } }
I20260807 18:06:31.811398 15565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } } }
I20260807 18:06:31.812266 15565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:31.812790 15558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 18:06:31.824179 15576 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:06:31.824371 15576 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:06:31.829381 15391 master_runner.cc:428] Detected that this master 8d8e31cd37f34c1b89577c57f1c3e014 is joining an existing cluster
I20260807 18:06:31.829545 15391 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.62.188:38419
I20260807 18:06:31.846191 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14880
I20260807 18:06:31.857615 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.835179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.8.62.190:44939)
0807 18:06:31.835443 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.835450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.836348 (+ 898us) server_negotiation.cc:407] Connection header received
0807 18:06:31.836699 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.836707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.836808 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.836886 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.841079 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.842436 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.844772 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.845486 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.845591 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.849158 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.849168 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.849173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.849219 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.853748 (+ 4529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.854851 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.854858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.854862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.855045 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.855604 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.855609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.855613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.856039 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.856191 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.856907 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.857194 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 18:06:31.857915 15582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.835186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44610)
0807 18:06:31.836187 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.836208 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.836480 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.837129 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.837135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.840223 (+ 3088us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.840782 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.840807 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.842685 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.842690 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.844585 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.844600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.845908 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.845914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.846160 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.847131 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.847153 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.848800 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.853981 (+ 5181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.854003 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.854009 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.854692 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.855196 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.855199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.855203 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.855394 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.856276 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.856280 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.856519 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.856735 (+ 216us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.857130 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":64,"thread_start_us":636,"threads_started":1}
I20260807 18:06:31.858630 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.836607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.8.62.189:37593)
0807 18:06:31.836861 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.836870 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.838423 (+ 1553us) server_negotiation.cc:407] Connection header received
0807 18:06:31.838757 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.838764 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.838855 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.838936 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.841741 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.843058 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.845725 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.846509 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.846604 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.850238 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.850255 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.850264 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.850322 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.854340 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.855200 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.855205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.855210 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.855368 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.855918 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.855923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.855927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.856294 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.856424 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.857198 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.857513 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:06:31.860671 15583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.836381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39940)
0807 18:06:31.838310 (+ 1929us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.838328 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.838507 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.839320 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.839328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.840667 (+ 1339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.841522 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.841547 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.843310 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.843315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.845549 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.845562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.847134 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.847141 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.847507 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.848303 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.848326 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.849837 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.854528 (+ 4691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.854535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.854542 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.855051 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.855604 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.855609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.855613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.855784 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.856512 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.856521 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.856898 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.859371 (+ 2473us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.859779 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1695,"mutex_wait_us":1,"thread_start_us":1289,"threads_started":1}
I20260807 18:06:31.875394 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.838751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.8.62.188:38419)
0807 18:06:31.838936 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.838941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.839992 (+ 1051us) server_negotiation.cc:407] Connection header received
0807 18:06:31.840212 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.840222 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.840288 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.840376 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.842806 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.844289 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.858889 (+ 14600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.859842 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.859963 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.867074 (+ 7111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.867083 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.867091 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.867144 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.871391 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.872666 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.872677 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.872681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.872850 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.873276 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.873281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.873286 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.873673 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.873850 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.874464 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.874733 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":64}
I20260807 18:06:31.877068 15584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.838349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.0.0.1:46756)
0807 18:06:31.839889 (+ 1540us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.839909 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.840085 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.840659 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.840664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.841895 (+ 1231us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.842623 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.842640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.854040 (+ 11400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.854046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.857495 (+ 3449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.857518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.860381 (+ 2863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.860387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.860738 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.862353 (+ 1615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.862389 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.866817 (+ 4428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.871638 (+ 4821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.871652 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.871656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.872484 (+ 828us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.872974 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.872979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.872983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.873149 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.873921 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.873928 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.874183 (+ 255us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.875689 (+ 1506us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.876103 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1395,"thread_start_us":966,"threads_started":1}
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125988, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.190@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
Aug 07 18:06:31 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.129@KRBTEST.COM
I20260807 18:06:31.890296 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.865698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46758 (local address 127.8.62.188:38419)
0807 18:06:31.868591 (+ 2893us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.868597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.868623 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:06:31.868731 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.868738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.868829 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.870293 (+ 1464us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.871303 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.873183 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.875572 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.876504 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.877515 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.881780 (+ 4265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.881788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.881793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.881843 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.886536 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.887660 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.887666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.887670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.887880 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.888378 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.888383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.888386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.888863 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.889011 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.889599 (+ 588us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.889841 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2717,"thread_start_us":2796,"threads_started":1}
I20260807 18:06:31.890280 15501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.865337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.0.0.1:46758)
0807 18:06:31.865547 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:31.865570 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:31.865698 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:31.869204 (+ 3506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:31.869210 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:31.869769 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:31.870030 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.870051 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.873377 (+ 3326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.873385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.875208 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:31.875245 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.876812 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.876819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:31.877089 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.877852 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:31.877879 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:31.881383 (+ 3504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:31.886754 (+ 5371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.886761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.886765 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.887454 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.888054 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:31.888059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:31.888063 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:31.888240 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:31.889026 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:31.889031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:31.889274 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 18:06:31.889449 (+ 175us) client_negotiation.cc:240] Negotiation successful
0807 18:06:31.889801 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:06:31.892087 14816 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.62.188:38419: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
member_type: NON_VOTER
last_known_addr {
host: "127.8.62.188"
port: 38419
}
attrs {
promote: true
}
}
dest_uuid: "9e0af4e45b0e4e4196365f0b3144e996"
cas_config_opid_index: 5
I20260807 18:06:31.893319 14816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } }
I20260807 18:06:31.894241 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:31.847680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37736 (local address 127.8.62.129:41749)
0807 18:06:31.865604 (+ 17924us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:31.865614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:31.865647 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:06:31.865722 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:31.865731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:31.866087 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:31.866390 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:31.869703 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.873183 (+ 3480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.875447 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:31.876504 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:31.877706 (+ 1202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:31.884880 (+ 7174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:31.884907 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:31.884930 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:31.884984 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:31.889789 (+ 4805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.891317 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.891328 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.891331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.891538 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:31.892188 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:31.892193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:31.892197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:31.892731 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:31.893067 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:31.893457 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:06:31.893782 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17706,"thread_start_us":17549,"threads_started":1}
I20260807 18:06:31.895429 15500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.901396 15007 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:31.902287 15531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:31.903120 15444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:06:31.910813 15531 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 18:06:31.912197 15587 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260807 18:06:31.914575 15500 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.62.188:38419
I20260807 18:06:31.915412 15391 master.cc:537] Master@127.8.62.188:38419 shutting down...
I20260807 18:06:31.914942 15590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 18:06:31.919668 15589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.920157 15589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
W20260807 18:06:31.920233 15510 tablet.cc:2369] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:31.926676 15444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.943487 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: NON_VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: true } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.943921 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: LEARNER
W20260807 18:06:31.949165 14801 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 -> Peer 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188:38419): Couldn't send request to peer 8d8e31cd37f34c1b89577c57f1c3e014. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:06:31.949254 15589 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: attempting to promote NON_VOTER 5ea4e7b4593a4c3d80fe664c2fffb6fb to VOTER
I20260807 18:06:31.950213 15589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } }
I20260807 18:06:31.952782 15444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 LEARNER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 18:06:31.953604 15589 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260807 18:06:31.954236 14801 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 -> Peer 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188:38419): Couldn't send request to peer 8d8e31cd37f34c1b89577c57f1c3e014. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 18:06:31.956446 15500 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.959336 15500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.959652 15500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:31.959589 15587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.959939 15587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:31.960722 15444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.963263 15391 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:31.963701 15391 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:31.964164 15391 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: stopping tablet replica
I20260807 18:06:31.964381 15474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:31.964675 15474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:31.972404 15391 master.cc:552] Master@127.8.62.188:38419 shutdown complete.
I20260807 18:06:31.972602 15391 master_runner.cc:305] Clearing existing system tablet
I20260807 18:06:31.973235 15391 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:31.973418 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:31.977308 15391 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 18:06:31.980969 15596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:31.981455 15391 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:31.981745 15391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:06:31.981863 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
format_stamp: "Formatted at 2026-08-07 18:06:31 on dist-test-slave-rnrw"
server_key: "135b9785827c1d08ab2f3a325eb9225d"
server_key_iv: "996faf03c6b8d45b6123cea47afd64bc"
server_key_version: "encryptionkey@0"
I20260807 18:06:31.987327 15391 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:31.995082 15391 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 18:06:31.995281 15391 log.cc:1196] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 18:06:31.995769 15391 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Deleting consensus metadata
I20260807 18:06:31.996030 15391 master_runner.cc:315] Copying system tablet from 127.8.62.190:44939
I20260807 18:06:32.002578 15391 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: tablet copy: Beginning tablet copy session from remote peer at address 127.8.62.190:44939
I20260807 18:06:32.023382 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.003620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.8.62.190:44939)
0807 18:06:32.003896 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.003904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.005492 (+ 1588us) server_negotiation.cc:407] Connection header received
0807 18:06:32.005773 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.005782 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.005879 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.006007 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:32.007323 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.008598 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.010942 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.011728 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.011894 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.015217 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:32.015224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:32.015230 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:32.015283 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:32.019579 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.020663 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.020669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.020673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.020883 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.021463 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.021468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.021471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.021896 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:32.022051 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.022706 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.022958 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260807 18:06:32.023703 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.003515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:44620)
0807 18:06:32.005341 (+ 1826us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:32.005362 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:32.005600 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:32.006049 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:32.006054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:32.006682 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:32.007143 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.007158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.008755 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.008761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.010739 (+ 1978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.010757 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.011968 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.011975 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.012296 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.013270 (+ 974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:32.013297 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:32.014900 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:32.019753 (+ 4853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.019760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.019765 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.020452 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.021014 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.021020 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.021024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.021320 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.022044 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:32.022049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:32.022388 (+ 339us) client_negotiation.cc:769] Sending connection context
0807 18:06:32.022531 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 18:06:32.023022 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1624,"thread_start_us":1344,"threads_started":1}
I20260807 18:06:32.024757 14837 tablet_copy_service.cc:140] P 9e0af4e45b0e4e4196365f0b3144e996: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8d8e31cd37f34c1b89577c57f1c3e014 ({username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:44620)
I20260807 18:06:32.024962 14837 tablet_copy_service.cc:161] P 9e0af4e45b0e4e4196365f0b3144e996: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8d8e31cd37f34c1b89577c57f1c3e014 at {username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:44620: session id = 8d8e31cd37f34c1b89577c57f1c3e014-00000000000000000000000000000000
I20260807 18:06:32.029858 14837 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 18:06:32.032214 15391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 18:06:32.041033 15391 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: tablet copy: Starting download of 0 data blocks...
I20260807 18:06:32.041616 15391 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: tablet copy: Starting download of 1 WAL segments...
I20260807 18:06:32.045552 15391 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 18:06:32.051198 14837 tablet_copy_service.cc:342] P 9e0af4e45b0e4e4196365f0b3144e996: Request end of tablet copy session 8d8e31cd37f34c1b89577c57f1c3e014-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:44620
I20260807 18:06:32.051383 14837 tablet_copy_service.cc:433] P 9e0af4e45b0e4e4196365f0b3144e996: ending tablet copy session 8d8e31cd37f34c1b89577c57f1c3e014-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8d8e31cd37f34c1b89577c57f1c3e014
I20260807 18:06:32.066885 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14880
I20260807 18:06:32.067101 15391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:32.071863 15603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:32.077807 15604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:32.085012 15606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:32.086036 15391 server_base.cc:1034] running on GCE node
I20260807 18:06:32.086410 15391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:32.087116 15391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:32.088331 15391 hybrid_clock.cc:648] HybridClock initialized: now 1786125992088285 us; error 37 us; skew 500 ppm
I20260807 18:06:32.088625 15391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125992, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:32.091086 15391 init.cc:377] Logged in from keytab as oryx/127.8.62.188@KRBTEST.COM (short username oryx)
I20260807 18:06:32.093382 15391 webserver.cc:460] Webserver started at http://127.8.62.188:38081/ using document root <none> and password file <none>
I20260807 18:06:32.093669 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:41749
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:44939
--webserver_port=34355
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:06:32.093724 15391 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:32.093833 15391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:32.097122 15391 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260807 18:06:32.099325 15613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:32.099988 15391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260807 18:06:32.100126 15391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "8d8e31cd37f34c1b89577c57f1c3e014"
format_stamp: "Formatted at 2026-08-07 18:06:31 on dist-test-slave-rnrw"
server_key: "135b9785827c1d08ab2f3a325eb9225d"
server_key_iv: "996faf03c6b8d45b6123cea47afd64bc"
server_key_version: "encryptionkey@0"
I20260807 18:06:32.100328 15391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:32.129568 15391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:32.134116 15391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:32.172952 15391 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.188:38419
I20260807 18:06:32.172981 15666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.188:38419 every 8 connection(s)
I20260807 18:06:32.173916 15391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
I20260807 18:06:32.182138 15667 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:06:32.188377 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Bootstrap starting.
I20260807 18:06:32.207280 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:06:32.207895 15667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Bootstrap complete.
I20260807 18:06:32.209066 15667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } }
I20260807 18:06:32.209372 15667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8d8e31cd37f34c1b89577c57f1c3e014, State: Initialized, Role: LEARNER
I20260807 18:06:32.209654 15667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } }
I20260807 18:06:32.210538 15670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:32.210909 15667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:32.211048 15670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: LEARNER
I20260807 18:06:32.254923 15681 catalog_manager.cc:1259] Loaded cluster ID: b09efc591e7749d8a2c7f618beea1679
I20260807 18:06:32.255064 15681 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: loading cluster ID for follower catalog manager: success
I20260807 18:06:32.259063 15681 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: acquiring CA information for follower catalog manager: success
I20260807 18:06:32.261861 15681 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:06:32.542495 15614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:32.543056 15614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:32.543253 15614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:32.551997 15614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:32.552172 15614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:32.552348 15614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:32.565516 15614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:41749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=34355
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:32.566926 15614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:32.568434 15614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 18:06:32.576300 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.569191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.0.0.1:46764)
0807 18:06:32.569728 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:32.569750 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:32.569970 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:32.570647 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:32.570652 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:32.570681 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:32.570955 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.570973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.572753 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.572760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.573939 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.573951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.575359 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.575365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.575549 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.575589 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:06:32.575734 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 18:06:32.575918 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":293,"threads_started":1}
I20260807 18:06:32.576334 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.569348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.8.62.188:38419)
0807 18:06:32.570315 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.570320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.570337 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:06:32.570400 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.570404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.570473 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.570615 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:32.571129 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.572522 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.574116 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.575166 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.575341 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.575563 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.575775 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.575949 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":561,"threads_started":1}
I20260807 18:06:32.577613 15642 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 18:06:32.578316 15670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e0af4e45b0e4e4196365f0b3144e996. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:32.578720 15670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: LEARNER
W20260807 18:06:32.584980 15689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:32.585853 15692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:32.586199 15690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:32.590396 15614 server_base.cc:1034] running on GCE node
I20260807 18:06:32.591282 15614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:32.592494 15614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:32.593815 15614 hybrid_clock.cc:648] HybridClock initialized: now 1786125992593707 us; error 57 us; skew 500 ppm
I20260807 18:06:32.594226 15614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125992, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:32.598624 15614 init.cc:377] Logged in from keytab as oryx/127.8.62.129@KRBTEST.COM (short username oryx)
I20260807 18:06:32.600726 15614 webserver.cc:460] Webserver started at http://127.8.62.129:34355/ using document root <none> and password file <none>
I20260807 18:06:32.601904 15614 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:32.602088 15614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:32.608695 15614 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.005s
I20260807 18:06:32.612340 15699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:32.614120 15614 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 18:06:32.614405 15614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "58d35ba8e6f5c7f84b85a314346056f8"
server_key_iv: "00e85a24b5803b9f6d4c56e8bfceb766"
server_key_version: "encryptionkey@0"
I20260807 18:06:32.615629 15614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:32.637261 15614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:32.643153 15614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:32.643463 15614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:32.644830 15614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:32.646832 15614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:32.646926 15614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:32.647009 15614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:32.647075 15614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:32.696573 15614 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:41749
I20260807 18:06:32.696785 15814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:41749 every 8 connection(s)
I20260807 18:06:32.698375 15614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:32.702580 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15614
I20260807 18:06:32.702932 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15019
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125992, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125992, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125992, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
Aug 07 18:06:32 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.130@KRBTEST.COM
I20260807 18:06:32.745679 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.702166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:44737 (local address 127.8.62.188:38419)
0807 18:06:32.702407 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.702413 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.704190 (+ 1777us) server_negotiation.cc:407] Connection header received
0807 18:06:32.708125 (+ 3935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.708135 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.708264 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.708354 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:32.711638 (+ 3284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.713097 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.716837 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.717968 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.718091 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.734667 (+ 16576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:32.734676 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:32.734691 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:32.734740 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:32.740279 (+ 5539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.741843 (+ 1564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.741850 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.741854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.742134 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.742950 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.742957 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.742960 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.743487 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:32.743631 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.744900 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.745188 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:06:32.746937 15825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.718707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:46291 (local address 127.8.62.189:37593)
0807 18:06:32.719640 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.719647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.721606 (+ 1959us) server_negotiation.cc:407] Connection header received
0807 18:06:32.721691 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.721696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.721800 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.721907 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:32.724921 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.726006 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.727871 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.728295 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.728441 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.734558 (+ 6117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:32.734566 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:32.734572 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:32.734673 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:32.740347 (+ 5674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.742032 (+ 1685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.742040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.742045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.742325 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.742897 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.742905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.742909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.743429 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:32.743583 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.745525 (+ 1942us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.746253 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":598,"threads_started":1}
I20260807 18:06:32.747558 15823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.705076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:45777 (local address 127.8.62.190:44939)
0807 18:06:32.717174 (+ 12098us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.717183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.719285 (+ 2102us) server_negotiation.cc:407] Connection header received
0807 18:06:32.721358 (+ 2073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.721364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.721478 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.722524 (+ 1046us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:32.725755 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.727357 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.729880 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.730810 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.730948 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.739213 (+ 8265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:32.739230 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:32.739235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:32.739287 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:32.743448 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.744690 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.744698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.744700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.744955 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.745509 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.745514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.745517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.746166 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:32.746311 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.746862 (+ 551us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.747113 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11828,"thread_start_us":11709,"threads_started":1}
I20260807 18:06:32.748456 15820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.702911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.8.62.129:44737)
0807 18:06:32.704003 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:32.704064 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:32.707632 (+ 3568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:32.709136 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:32.709153 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:32.710141 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:32.711285 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.711319 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.713727 (+ 2408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.713737 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.716529 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.716610 (+ 81us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.721427 (+ 4817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.721437 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.721993 (+ 556us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.723341 (+ 1348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:32.723383 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:32.734238 (+ 10855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:32.740514 (+ 6276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.740526 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.740583 (+ 57us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.741656 (+ 1073us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.742314 (+ 658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.742321 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.742327 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.742797 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.743702 (+ 905us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:32.743719 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:32.744345 (+ 626us) client_negotiation.cc:769] Sending connection context
0807 18:06:32.746070 (+ 1725us) client_negotiation.cc:240] Negotiation successful
0807 18:06:32.746755 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":2,"spinlock_wait_cycles":8448,"thread_start_us":731,"threads_started":1}
I20260807 18:06:32.748826 15824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.705404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:45777)
0807 18:06:32.718663 (+ 13259us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:32.718693 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:32.721171 (+ 2478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:32.722806 (+ 1635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:32.722812 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:32.724773 (+ 1961us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:32.725470 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.725486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.727692 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.727696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.729643 (+ 1947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.729658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.731386 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.731400 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.731738 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.732621 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:32.732645 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:32.738836 (+ 6191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:32.743640 (+ 4804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.743646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.743652 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.744485 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.745129 (+ 644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.745134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.745138 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.745374 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.746393 (+ 1019us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:32.746397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:32.746577 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 18:06:32.747812 (+ 1235us) client_negotiation.cc:240] Negotiation successful
0807 18:06:32.748305 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12955,"mutex_wait_us":161,"thread_start_us":12571,"threads_started":1}
I20260807 18:06:32.749648 15826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.718721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.8.62.129:46291)
0807 18:06:32.721304 (+ 2583us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:32.721326 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:32.721445 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:32.722194 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:32.722199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:32.723413 (+ 1214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:32.724690 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.724706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.726172 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.726179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.727688 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:32.727702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.728589 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.728597 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:32.728801 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.729663 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:32.729682 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:32.734126 (+ 4444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:32.740606 (+ 6480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.740612 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.740615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.741830 (+ 1215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.742493 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:32.742499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:32.742504 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:32.742730 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:32.743741 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:32.743748 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:32.744294 (+ 546us) client_negotiation.cc:769] Sending connection context
0807 18:06:32.747481 (+ 3187us) client_negotiation.cc:240] Negotiation successful
0807 18:06:32.748133 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2433,"mutex_wait_us":5,"spinlock_wait_cycles":22528,"thread_start_us":2241,"threads_started":1}
I20260807 18:06:32.750773 15815 heartbeater.cc:347] Connected to a master server at 127.8.62.188:38419
I20260807 18:06:32.750916 15816 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:32.751171 15815 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:32.751202 15816 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:32.751878 15818 heartbeater.cc:347] Connected to a master server at 127.8.62.189:37593
I20260807 18:06:32.752002 15818 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:32.752214 15816 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
I20260807 18:06:32.752373 15818 heartbeater.cc:511] Master 127.8.62.189:37593 requested a full tablet report, sending...
I20260807 18:06:32.752799 15815 heartbeater.cc:511] Master 127.8.62.188:38419 requested a full tablet report, sending...
I20260807 18:06:32.754483 14816 ts_manager.cc:194] Re-registered known tserver with Master: ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:32.754925 15631 ts_manager.cc:194] Registered new tserver with Master: ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:32.754933 15434 ts_manager.cc:194] Registered new tserver with Master: ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:32.755820 14816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:45777
I20260807 18:06:32.756534 15822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:32.704065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52902 (local address 127.8.62.130:39541)
0807 18:06:32.716538 (+ 12473us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:32.716570 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:32.720334 (+ 3764us) server_negotiation.cc:407] Connection header received
0807 18:06:32.720810 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:32.720832 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:32.721247 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:32.721405 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:32.731543 (+ 10138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.733589 (+ 2046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.736612 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:32.737692 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:32.737825 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:32.746363 (+ 8538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:32.746384 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:32.746402 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:32.746489 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:32.752536 (+ 6047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.753661 (+ 1125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.753671 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.753674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.753869 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:32.754353 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:32.754358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:32.754361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:32.754837 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:32.755131 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:32.755790 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 18:06:32.756082 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12170,"thread_start_us":14760,"threads_started":1}
I20260807 18:06:32.758649 15143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:32.849903 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15019
W20260807 18:06:32.867609 14803 connection.cc:535] server connection from 127.8.62.130:39225 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:06:32.868441 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:39541
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:44939
--webserver_port=41907
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:06:33.157567 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: NON_VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: true } } }
I20260807 18:06:33.158035 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:33.160094 15682 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: attempting to promote NON_VOTER 8d8e31cd37f34c1b89577c57f1c3e014 to VOTER
I20260807 18:06:33.160914 15682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } }
I20260807 18:06:33.164012 15642 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 LEARNER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:06:33.164084 15444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Refusing update from remote peer 9e0af4e45b0e4e4196365f0b3144e996: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 18:06:33.164805 15830 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:06:33.165441 15829 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 18:06:33.169147 15830 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.169973 15444 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.173166 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.173519 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:33.173840 15682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.174088 15682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:33.174953 15836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.175391 15836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:33.177091 15830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.177358 15830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:33.178524 15642 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.185178 15837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9e0af4e45b0e4e4196365f0b3144e996" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:33.185684 15837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 18:06:33.237025 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:33.237344 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:33.237447 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:33.246286 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:33.246420 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:33.246541 15828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:33.258090 15828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:39541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=41907
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:33.259521 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:33.261004 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:33.275580 15842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:33.275632 15843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:33.275655 15845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:33.277123 15828 server_base.cc:1034] running on GCE node
I20260807 18:06:33.277935 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:33.279323 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:33.280668 15828 hybrid_clock.cc:648] HybridClock initialized: now 1786125993280589 us; error 62 us; skew 500 ppm
I20260807 18:06:33.281066 15828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:33.285223 15828 init.cc:377] Logged in from keytab as oryx/127.8.62.130@KRBTEST.COM (short username oryx)
I20260807 18:06:33.286998 15828 webserver.cc:460] Webserver started at http://127.8.62.130:41907/ using document root <none> and password file <none>
I20260807 18:06:33.287883 15828 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:33.288062 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:33.294914 15828 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:06:33.298486 15852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:33.300419 15828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:06:33.300746 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "055a3f0da5a74e959525874718da97bd"
format_stamp: "Formatted at 2026-08-07 18:06:29 on dist-test-slave-rnrw"
server_key: "e5b836458b5fbac5132a01a5484f5cc8"
server_key_iv: "678d2873aac713d7fb189e327ca7be87"
server_key_version: "encryptionkey@0"
I20260807 18:06:33.301986 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:33.316216 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:33.321600 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:33.321916 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:33.323148 15828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:33.324980 15828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:33.325116 15828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:33.325254 15828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:33.325350 15828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:33.365036 15828 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:39541
I20260807 18:06:33.365104 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:39541 every 8 connection(s)
I20260807 18:06:33.366807 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:33.373934 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15828
I20260807 18:06:33.374321 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15154
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.131@KRBTEST.COM
I20260807 18:06:33.402877 15972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.370075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:58631 (local address 127.8.62.189:37593)
0807 18:06:33.370598 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:33.370603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:33.371814 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 18:06:33.375012 (+ 3198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:33.375019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:33.375116 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:33.375244 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:33.379477 (+ 4233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.381092 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.386143 (+ 5051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.387137 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.387284 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.393544 (+ 6260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:33.393552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:33.393557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:33.393598 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:33.398179 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.399113 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.399121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.399126 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.399320 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.400562 (+ 1242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.400569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.400572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.400932 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:33.401121 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:33.402184 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0807 18:06:33.402479 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":220,"threads_started":1}
I20260807 18:06:33.404132 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.370688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.8.62.130:58631)
0807 18:06:33.371505 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:33.371647 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:33.374607 (+ 2960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:33.375694 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:33.375707 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:33.378743 (+ 3036us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:33.379272 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.379283 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.381642 (+ 2359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.381650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.385954 (+ 4304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.385969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.387743 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.387751 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.387982 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.388743 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:33.388766 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:33.393145 (+ 4379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:33.398363 (+ 5218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.398369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.398376 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.398926 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.400068 (+ 1142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.400072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.400075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.400381 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.401080 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:33.401089 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:33.401518 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 18:06:33.403187 (+ 1669us) client_negotiation.cc:240] Negotiation successful
0807 18:06:33.403487 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":2304,"thread_start_us":373,"threads_started":1}
I20260807 18:06:33.404640 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.372462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:59547 (local address 127.8.62.190:44939)
0807 18:06:33.375047 (+ 2585us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:33.375055 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:33.375078 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:06:33.375157 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:33.375163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:33.375245 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:33.375379 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:33.378642 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.380346 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.384388 (+ 4042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.385233 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.385366 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.392465 (+ 7099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:33.392471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:33.392474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:33.392517 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:33.396759 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.399539 (+ 2780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.399545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.399547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.399832 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.402856 (+ 3024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.402862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.402865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.403331 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:33.403515 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:33.403967 (+ 452us) server_negotiation.cc:299] Negotiation successful
0807 18:06:33.404227 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2451,"thread_start_us":888,"threads_started":1}
I20260807 18:06:33.404641 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.373912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.8.62.130:41503)
0807 18:06:33.379035 (+ 5123us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:33.379056 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:33.379258 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:33.380083 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:33.380088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:33.380885 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:33.381302 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.381314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.382900 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.382906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.384384 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.384398 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.385725 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.385731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.385959 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.387204 (+ 1245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:33.387265 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:33.392019 (+ 4754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:33.399441 (+ 7422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.399449 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.399456 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.399901 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.400370 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.400374 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.400377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.400707 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.401597 (+ 890us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:33.401601 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:33.401733 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 18:06:33.401964 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 18:06:33.402552 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4958,"spinlock_wait_cycles":20096,"thread_start_us":4995,"threads_started":1}
I20260807 18:06:33.405323 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.372233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:59547)
0807 18:06:33.373050 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:33.373078 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:33.374607 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:33.375694 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:33.375707 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:33.376777 (+ 1070us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:33.378298 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.378360 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.381074 (+ 2714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.381079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.384224 (+ 3145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:33.384245 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.386259 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.386269 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:33.386530 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.387216 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:33.387244 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:33.392205 (+ 4961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:33.396938 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.396949 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.396972 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.397764 (+ 792us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.402067 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:33.402071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:33.402076 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:33.402290 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:33.403490 (+ 1200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:33.403495 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:33.403645 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 18:06:33.403833 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 18:06:33.404124 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":112,"spinlock_wait_cycles":2944,"thread_start_us":465,"threads_started":1}
I20260807 18:06:33.406231 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.373331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:41503 (local address 127.8.62.188:38419)
0807 18:06:33.373976 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:33.373984 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:33.379393 (+ 5409us) server_negotiation.cc:407] Connection header received
0807 18:06:33.379474 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:33.379478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:33.379569 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:33.379806 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:33.381477 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.382736 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.384609 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.385247 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.385366 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.392386 (+ 7020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:33.392395 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:33.392402 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:33.392450 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:33.399309 (+ 6859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.400058 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.400064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.400067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.400220 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.400859 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.400866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.400869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.401379 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:33.401621 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:33.404221 (+ 2600us) server_negotiation.cc:299] Negotiation successful
0807 18:06:33.405774 (+ 1553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":346,"threads_started":1}
I20260807 18:06:33.406239 15969 heartbeater.cc:347] Connected to a master server at 127.8.62.189:37593
I20260807 18:06:33.406363 15968 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:33.406616 15968 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:33.406620 15969 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:33.407464 15967 heartbeater.cc:347] Connected to a master server at 127.8.62.188:38419
I20260807 18:06:33.407614 15967 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:33.407846 15968 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
I20260807 18:06:33.408214 15979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:33.375881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.8.62.131:33865)
0807 18:06:33.378000 (+ 2119us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:33.378009 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:33.381114 (+ 3105us) server_negotiation.cc:407] Connection header received
0807 18:06:33.381587 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:33.381598 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:33.381918 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:33.382244 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:33.384360 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.386179 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.388343 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:33.389537 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:33.389662 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:33.397279 (+ 7617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:33.397301 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:33.397322 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:33.397374 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:33.401844 (+ 4470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.402819 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.402830 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.402832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.403321 (+ 489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:33.405813 (+ 2492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:33.405818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:33.405822 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:33.406264 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:33.406659 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:33.407367 (+ 708us) server_negotiation.cc:299] Negotiation successful
0807 18:06:33.407717 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1343,"thread_start_us":1008,"threads_started":1}
I20260807 18:06:33.408253 15967 heartbeater.cc:511] Master 127.8.62.188:38419 requested a full tablet report, sending...
I20260807 18:06:33.409498 15969 heartbeater.cc:511] Master 127.8.62.189:37593 requested a full tablet report, sending...
I20260807 18:06:33.409920 15279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:33.410104 14816 ts_manager.cc:194] Re-registered known tserver with Master: 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541)
I20260807 18:06:33.411010 15434 ts_manager.cc:194] Registered new tserver with Master: 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541)
I20260807 18:06:33.411330 14816 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:59547
I20260807 18:06:33.412401 15631 ts_manager.cc:194] Registered new tserver with Master: 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541)
I20260807 18:06:33.498822 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15154
I20260807 18:06:33.518925 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:33865
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:44939
--webserver_port=42357
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:06:33.758728 15816 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
W20260807 18:06:33.874670 15981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:33.875089 15981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:33.875283 15981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:33.883544 15981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:33.883678 15981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:33.883855 15981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:33.894434 15981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:33865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=42357
--tserver_master_addrs=127.8.62.190:44939,127.8.62.189:37593,127.8.62.188:38419
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:33.895957 15981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:33.897540 15981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:33.913817 15987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:33.913863 15990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:33.914108 15988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:33.914871 15981 server_base.cc:1034] running on GCE node
I20260807 18:06:33.915599 15981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:33.916744 15981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:33.918040 15981 hybrid_clock.cc:648] HybridClock initialized: now 1786125993917990 us; error 43 us; skew 500 ppm
I20260807 18:06:33.918406 15981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:33 dist-test-slave-rnrw krb5kdc[14771](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:33.922333 15981 init.cc:377] Logged in from keytab as oryx/127.8.62.131@KRBTEST.COM (short username oryx)
I20260807 18:06:33.924010 15981 webserver.cc:460] Webserver started at http://127.8.62.131:42357/ using document root <none> and password file <none>
I20260807 18:06:33.924959 15981 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:33.925133 15981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:33.931116 15981 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:06:33.934245 15997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:33.935730 15981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:06:33.935930 15981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "5948fd37fba0439fb7ff451085ef9d49"
format_stamp: "Formatted at 2026-08-07 18:06:30 on dist-test-slave-rnrw"
server_key: "9cb59ded353eb7b6b1caca10cdd7cec2"
server_key_iv: "5305ccfd234ca39a34226aa3a9555ba7"
server_key_version: "encryptionkey@0"
I20260807 18:06:33.937047 15981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:33.954985 15981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:33.959944 15981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:33.960250 15981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:33.961427 15981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:33.962852 15981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:33.962980 15981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:33.963119 15981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:33.963205 15981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:34.000859 15981 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:33865
I20260807 18:06:34.000907 16112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:33865 every 8 connection(s)
I20260807 18:06:34.002485 15981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:34.003484 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 15981
Aug 07 18:06:34 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.188@KRBTEST.COM
I20260807 18:06:34.051751 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.006976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41983 (local address 127.8.62.188:38419)
0807 18:06:34.008285 (+ 1309us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.008295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.008611 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:06:34.012599 (+ 3988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.012610 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.012764 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.012915 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.016394 (+ 3479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.018050 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.021387 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.022097 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.022244 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.037996 (+ 15752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.038004 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.038009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.038059 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.042722 (+ 4663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.044887 (+ 2165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.044893 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.044899 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.045207 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.046526 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.046532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.046535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.046989 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.047122 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.050614 (+ 3492us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.051216 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1128,"thread_start_us":1015,"threads_started":1}
Aug 07 18:06:34 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.189@KRBTEST.COM
Aug 07 18:06:34 dist-test-slave-rnrw krb5kdc[14771](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125993, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:34.055521 16121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.007164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.8.62.131:41983)
0807 18:06:34.008180 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.008239 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.011992 (+ 3753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.013807 (+ 1815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.013827 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.014922 (+ 1095us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:34.016063 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.016099 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.018381 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.018390 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.021187 (+ 2797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.021200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.022704 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.022711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.023079 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.030077 (+ 6998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:34.030129 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:34.036050 (+ 5921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:34.042976 (+ 6926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.042985 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.043005 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.043763 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.045420 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.045424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.045428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.045836 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.047264 (+ 1428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:34.047273 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:34.047908 (+ 635us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.052617 (+ 4709us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.053342 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":450,"threads_started":1}
I20260807 18:06:34.059024 16113 heartbeater.cc:347] Connected to a master server at 127.8.62.188:38419
I20260807 18:06:34.059433 16113 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:34.060670 16113 heartbeater.cc:511] Master 127.8.62.188:38419 requested a full tablet report, sending...
I20260807 18:06:34.063251 15631 ts_manager.cc:194] Registered new tserver with Master: 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865)
I20260807 18:06:34.066330 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.028512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.8.62.131:41655)
0807 18:06:34.036098 (+ 7586us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.036123 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.036454 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.040395 (+ 3941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.040402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.041093 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:34.041496 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.041511 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.045863 (+ 4352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.045871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.047621 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.047634 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.050379 (+ 2745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.050391 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.050577 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.051580 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:34.051604 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:34.055142 (+ 3538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:34.061792 (+ 6650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.061799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.061803 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.062816 (+ 1013us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.063425 (+ 609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.063442 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.063446 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.063788 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.064902 (+ 1114us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:34.064907 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:34.065080 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.065214 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.065560 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7438,"thread_start_us":9398,"threads_started":1}
I20260807 18:06:34.067843 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.021760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41655 (local address 127.8.62.189:37593)
0807 18:06:34.039487 (+ 17727us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.039493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.039514 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:34.039587 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.039594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.039698 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.040018 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.043802 (+ 3784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.045659 (+ 1857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.048464 (+ 2805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.049446 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.049585 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.055464 (+ 5879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.055472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.055486 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.055541 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.061592 (+ 6051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.062987 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.062993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.062997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.063260 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.063960 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.063966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.063978 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.064611 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.064819 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.066797 (+ 1978us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.067143 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17565,"thread_start_us":18400,"threads_started":1}
I20260807 18:06:34.068926 16115 heartbeater.cc:347] Connected to a master server at 127.8.62.189:37593
I20260807 18:06:34.069125 16115 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:34.069705 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.009225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:49533 (local address 127.8.62.190:44939)
0807 18:06:34.035894 (+ 26669us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.035905 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.035934 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:06:34.036026 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.036035 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.036151 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.036464 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.037995 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.039429 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.043958 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.045023 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.045182 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.057762 (+ 12580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.057769 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.057774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.057891 (+ 117us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.063230 (+ 5339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.064508 (+ 1278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.064517 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.064524 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.064747 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.065292 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.065300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.065308 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.065773 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.066010 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.068522 (+ 2512us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.068919 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26459,"thread_start_us":31163,"threads_started":1}
I20260807 18:06:34.069690 16115 heartbeater.cc:511] Master 127.8.62.189:37593 requested a full tablet report, sending...
I20260807 18:06:34.070698 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.009040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:49533)
0807 18:06:34.020838 (+ 11798us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.020862 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.021054 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.036645 (+ 15591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.036653 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.037433 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:34.037847 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.037864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.041736 (+ 3872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.041740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.043579 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.043598 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.047875 (+ 4277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.047896 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.048095 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.049899 (+ 1804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:34.049927 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:34.057490 (+ 7563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:34.063555 (+ 6065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.063561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.063565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.064287 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.064928 (+ 641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:34.064934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:34.064938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:34.065130 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:34.066112 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:34.066119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:34.066379 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.067281 (+ 902us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.067715 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11636,"thread_start_us":12084,"threads_started":1}
I20260807 18:06:34.071105 15434 ts_manager.cc:194] Registered new tserver with Master: 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865)
I20260807 18:06:34.072597 16114 heartbeater.cc:347] Connected to a master server at 127.8.62.190:44939
I20260807 18:06:34.072806 16114 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:34.073396 16124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.010417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.8.62.190:44939)
0807 18:06:34.040573 (+ 30156us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.040582 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.040607 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:06:34.040669 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.040677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.040778 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.046095 (+ 5317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.046344 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.047856 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.050386 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.051287 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.051573 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.056289 (+ 4716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.056299 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.056305 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.056357 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.067730 (+ 11373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.069175 (+ 1445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.069181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.069185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.069356 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.069963 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.069969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.069972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.070423 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.071160 (+ 737us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.072259 (+ 1099us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.072833 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29815,"thread_start_us":26294,"threads_started":1}
I20260807 18:06:34.073436 16114 heartbeater.cc:511] Master 127.8.62.190:44939 requested a full tablet report, sending...
I20260807 18:06:34.074764 14815 ts_manager.cc:194] Re-registered known tserver with Master: 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865)
I20260807 18:06:34.076328 14815 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:49533
I20260807 18:06:34.084385 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.023893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.8.62.189:37593)
0807 18:06:34.049648 (+ 25755us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.049654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.049671 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:06:34.049732 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.049740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.049824 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.049971 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.063789 (+ 13818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.065508 (+ 1719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.067729 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.068502 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.068639 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.075794 (+ 7155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.075804 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.075809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.075854 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.080354 (+ 4500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.081483 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.081489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.081495 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.081689 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.082204 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.082209 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.082214 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.082611 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.082901 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.083615 (+ 714us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.083864 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25543,"thread_start_us":26393,"threads_started":1}
I20260807 18:06:34.085963 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.032016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.8.62.188:38419)
0807 18:06:34.052037 (+ 20021us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.052043 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.052070 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:06:34.052128 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.052134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.052258 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.052361 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.059765 (+ 7404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.061170 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.069836 (+ 8666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.070497 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.070618 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.077781 (+ 7163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.077790 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.077794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.077844 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.081692 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.082591 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.082598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.082603 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.082775 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.083270 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.083278 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.083282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.083671 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.084019 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.085188 (+ 1169us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.085440 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19896,"thread_start_us":24315,"threads_started":1}
I20260807 18:06:34.079790 14815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:34.090732 14815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:34.138976 16143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.114726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37738)
0807 18:06:34.127585 (+ 12859us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.127629 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.127976 (+ 347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.129326 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.129331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.129359 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:34.129798 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.129817 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.133545 (+ 3728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.133550 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.135752 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.135769 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.137742 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.137751 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.138020 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.138059 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.138134 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.138289 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12598,"mutex_wait_us":79,"thread_start_us":490,"threads_started":1}
I20260807 18:06:34.139971 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.115222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37738 (local address 127.8.62.129:41749)
0807 18:06:34.116533 (+ 1311us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.116572 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.128386 (+ 11814us) server_negotiation.cc:407] Connection header received
0807 18:06:34.128517 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.128525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.128925 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.129352 (+ 427us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:34.130062 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.133336 (+ 3274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.135989 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.137375 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.139292 (+ 1917us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.139523 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.139585 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.139727 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":893,"threads_started":1}
I20260807 18:06:34.140594 16145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.114093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:39541 (local address 127.0.0.1:52910)
0807 18:06:34.115531 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.115558 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.115879 (+ 321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.130719 (+ 14840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.130727 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.130768 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:34.131345 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.131360 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.133723 (+ 2363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.133732 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.137274 (+ 3542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.137286 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.139425 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.139432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.139683 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.139725 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.139822 (+ 97us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.139968 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"thread_start_us":13993,"threads_started":1}
I20260807 18:06:34.140516 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.114333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52910 (local address 127.8.62.130:39541)
0807 18:06:34.129667 (+ 15334us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.129680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.129723 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:06:34.129855 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.129867 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.130215 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.130718 (+ 503us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:34.131593 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.133524 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.137566 (+ 4042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.139226 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.139378 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.139690 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.139867 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.140118 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15067,"thread_start_us":16618,"threads_started":1}
I20260807 18:06:34.145390 15900 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 (DEFAULT_TABLE table=test-table [id=9ac2bddc29b84f37b9bba1d1d412b9fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:34.145467 15749 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 (DEFAULT_TABLE table=test-table [id=9ac2bddc29b84f37b9bba1d1d412b9fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:34.147200 15749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7acb18668b4c43b2871a8a5aa2eee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:34.147200 15900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7acb18668b4c43b2871a8a5aa2eee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:34.150144 16147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.116914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:33865 (local address 127.0.0.1:52858)
0807 18:06:34.136838 (+ 19924us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.136860 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.137131 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.139772 (+ 2641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.139780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.140891 (+ 1111us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:34.141631 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.141643 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.143633 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.143651 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.146366 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.146381 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.149001 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.149010 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.149271 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.149301 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.149426 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.149623 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19688,"thread_start_us":20330,"threads_started":1}
I20260807 18:06:34.150184 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.117331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.8.62.131:33865)
0807 18:06:34.132906 (+ 15575us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.132915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.137088 (+ 4173us) server_negotiation.cc:407] Connection header received
0807 18:06:34.137383 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.137394 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.137715 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.138075 (+ 360us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:34.141846 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.143481 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.146878 (+ 3397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.148725 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.148900 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.149191 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.149537 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.149742 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15361,"thread_start_us":15590,"threads_started":1}
I20260807 18:06:34.156013 16046 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 (DEFAULT_TABLE table=test-table [id=9ac2bddc29b84f37b9bba1d1d412b9fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:34.157611 16046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7acb18668b4c43b2871a8a5aa2eee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:34.168231 16150 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Bootstrap starting.
I20260807 18:06:34.169652 16149 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Bootstrap starting.
I20260807 18:06:34.175050 16149 tablet_bootstrap.cc:654] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:34.175197 16150 tablet_bootstrap.cc:654] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:34.177035 16149 log.cc:824] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:34.177139 16150 log.cc:824] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:34.191427 16149 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: No bootstrap required, opened a new log
I20260807 18:06:34.192108 16149 ts_tablet_manager.cc:1397] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.014s
I20260807 18:06:34.198539 16151 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Bootstrap starting.
I20260807 18:06:34.201910 16150 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: No bootstrap required, opened a new log
I20260807 18:06:34.202309 16150 ts_tablet_manager.cc:1397] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.012s
I20260807 18:06:34.204384 16151 tablet_bootstrap.cc:654] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:34.206382 16151 log.cc:824] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:34.208247 16149 raft_consensus.cc:348] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.208896 16149 raft_consensus.cc:374] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:34.209021 16149 raft_consensus.cc:729] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2508eaa73c4d0f95e060d1f5e3aedb, State: Initialized, Role: FOLLOWER
I20260807 18:06:34.209761 16149 consensus_queue.cc:260] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.210717 16151 tablet_bootstrap.cc:492] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: No bootstrap required, opened a new log
I20260807 18:06:34.212591 16151 ts_tablet_manager.cc:1397] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260807 18:06:34.216017 16150 raft_consensus.cc:348] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.216845 16150 raft_consensus.cc:374] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:34.217032 16150 raft_consensus.cc:729] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055a3f0da5a74e959525874718da97bd, State: Initialized, Role: FOLLOWER
I20260807 18:06:34.218007 16150 consensus_queue.cc:260] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.219883 15968 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
I20260807 18:06:34.224800 16150 ts_tablet_manager.cc:1428] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Time spent starting tablet: real 0.022s user 0.015s sys 0.004s
I20260807 18:06:34.229643 16151 raft_consensus.cc:348] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.230101 16151 raft_consensus.cc:374] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:34.230242 16151 raft_consensus.cc:729] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5948fd37fba0439fb7ff451085ef9d49, State: Initialized, Role: FOLLOWER
I20260807 18:06:34.230917 16151 consensus_queue.cc:260] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.232756 16114 heartbeater.cc:503] Master 127.8.62.190:44939 was elected leader, sending a full tablet report...
I20260807 18:06:34.233774 16151 ts_tablet_manager.cc:1428] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Time spent starting tablet: real 0.021s user 0.008s sys 0.007s
I20260807 18:06:34.234404 16149 ts_tablet_manager.cc:1428] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Time spent starting tablet: real 0.042s user 0.016s sys 0.000s
W20260807 18:06:34.258211 16119 tablet.cc:2369] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:06:34.370800 15970 tablet.cc:2369] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:34.417191 16157 raft_consensus.cc:482] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:34.417604 16157 raft_consensus.cc:504] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.419801 16157 leader_election.cc:288] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541), ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:34.427106 16130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.420302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:39541 (local address 127.8.62.131:44833)
0807 18:06:34.420727 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.420746 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.420917 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.421294 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.421302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.421325 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:34.421715 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.421730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.422862 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.422867 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.424758 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.424776 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.426052 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.426058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.426307 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.426359 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.426467 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.426749 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":265}
I20260807 18:06:34.427160 16144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.420400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44833 (local address 127.8.62.130:39541)
0807 18:06:34.420652 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.420658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.420860 (+ 202us) server_negotiation.cc:407] Connection header received
0807 18:06:34.421056 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.421062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.421163 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.421254 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:34.421900 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.422675 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.424947 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.425830 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.425992 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.426051 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.426577 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.426811 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:06:34.428243 16123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.420302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.8.62.131:35057)
0807 18:06:34.420498 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:34.420523 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:34.420690 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:34.421082 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:34.421085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:34.421103 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:34.421379 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.421389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.422886 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.422889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.424648 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:34.424663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.426541 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.426548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:34.427165 (+ 617us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.427650 (+ 485us) client_negotiation.cc:769] Sending connection context
0807 18:06:34.427751 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 18:06:34.427951 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260807 18:06:34.428411 16146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.420237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:35057 (local address 127.8.62.129:41749)
0807 18:06:34.420524 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.420530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.420710 (+ 180us) server_negotiation.cc:407] Connection header received
0807 18:06:34.420860 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.420866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.420947 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.421062 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:34.421608 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.422678 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.424884 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.426279 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.426491 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.426566 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.427824 (+ 1258us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.427933 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:06:34.428781 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7acb18668b4c43b2871a8a5aa2eee9" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055a3f0da5a74e959525874718da97bd" is_pre_election: true
I20260807 18:06:34.429394 15920 raft_consensus.cc:2454] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 0.
I20260807 18:06:34.429488 15769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7acb18668b4c43b2871a8a5aa2eee9" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" is_pre_election: true
I20260807 18:06:34.430066 15769 raft_consensus.cc:2454] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 0.
I20260807 18:06:34.430495 15998 leader_election.cc:302] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055a3f0da5a74e959525874718da97bd, 5948fd37fba0439fb7ff451085ef9d49; no voters:
I20260807 18:06:34.431071 16157 raft_consensus.cc:2781] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:34.431290 16157 raft_consensus.cc:482] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:34.431427 16157 raft_consensus.cc:3037] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:34.433972 16157 raft_consensus.cc:504] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.434854 16157 leader_election.cc:288] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 election: Requested vote from peers 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541), ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:34.435489 15769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7acb18668b4c43b2871a8a5aa2eee9" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
I20260807 18:06:34.435683 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7acb18668b4c43b2871a8a5aa2eee9" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055a3f0da5a74e959525874718da97bd"
I20260807 18:06:34.435810 15769 raft_consensus.cc:3037] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:34.436017 15920 raft_consensus.cc:3037] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:34.439687 15769 raft_consensus.cc:2454] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 1.
I20260807 18:06:34.439697 15920 raft_consensus.cc:2454] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 1.
I20260807 18:06:34.440258 16001 leader_election.cc:302] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5948fd37fba0439fb7ff451085ef9d49, ce2508eaa73c4d0f95e060d1f5e3aedb; no voters:
I20260807 18:06:34.440789 16157 raft_consensus.cc:2781] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:34.441509 16157 raft_consensus.cc:686] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 LEADER]: Becoming Leader. State: Replica: 5948fd37fba0439fb7ff451085ef9d49, State: Running, Role: LEADER
I20260807 18:06:34.442354 16157 consensus_queue.cc:237] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:34.448953 14816 catalog_manager.cc:5495] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "5948fd37fba0439fb7ff451085ef9d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } health_report { overall_health: HEALTHY } } }
W20260807 18:06:34.453689 15819 tablet.cc:2369] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:34.502786 16148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.496731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52874 (local address 127.8.62.131:33865)
0807 18:06:34.496995 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.496999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.497043 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:06:34.497251 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.497254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.497395 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.497532 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:34.497857 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.498833 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.500345 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.501042 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.501151 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.501652 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:34.502249 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:34.502361 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.502505 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.502578 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260807 18:06:34.512777 15769 raft_consensus.cc:1261] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Refusing update from remote peer 5948fd37fba0439fb7ff451085ef9d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:34.512777 15920 raft_consensus.cc:1261] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Refusing update from remote peer 5948fd37fba0439fb7ff451085ef9d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:34.513916 16158 consensus_queue.cc:1035] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:34.514355 16157 consensus_queue.cc:1035] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:34.531906 16163 mvcc.cc:205] Tried to move back new op lower bound from 7315972073511071744 to 7315972073240965120. Current Snapshot: MvccSnapshot[applied={T|T < 7315972073511071744}]
I20260807 18:06:34.555941 14816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:34.556388 14816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:34.566511 14816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9e0af4e45b0e4e4196365f0b3144e996: Sending DeleteTablet for 3 replicas of tablet 3b7acb18668b4c43b2871a8a5aa2eee9
I20260807 18:06:34.568017 15900 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:34 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52910
I20260807 18:06:34.568315 16046 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:34 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52858
I20260807 18:06:34.568526 15749 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b7acb18668b4c43b2871a8a5aa2eee9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:34 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:37738
I20260807 18:06:34.579625 16173 tablet_replica.cc:330] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: stopping tablet replica
I20260807 18:06:34.581951 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 14784
I20260807 18:06:34.583742 14849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:34.584481 16173 raft_consensus.cc:2229] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:34.586122 16173 raft_consensus.cc:2258] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:34.588369 16171 tablet_replica.cc:330] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: stopping tablet replica
I20260807 18:06:34.589016 16171 raft_consensus.cc:2229] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:34.589903 16171 raft_consensus.cc:2258] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:34.591890 16172 tablet_replica.cc:330] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: stopping tablet replica
I20260807 18:06:34.592269 16173 ts_tablet_manager.cc:1905] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:34.597034 16171 ts_tablet_manager.cc:1905] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:34.597934 16172 raft_consensus.cc:2229] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:34.598984 16172 raft_consensus.cc:2258] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:34.602860 16172 ts_tablet_manager.cc:1905] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:34.609555 16171 ts_tablet_manager.cc:1918] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:34.609863 16171 log.cc:1196] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/3b7acb18668b4c43b2871a8a5aa2eee9
I20260807 18:06:34.610780 16171 ts_tablet_manager.cc:1939] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 055a3f0da5a74e959525874718da97bd: Deleting consensus metadata
I20260807 18:06:34.611800 16173 ts_tablet_manager.cc:1918] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:34.612130 16173 log.cc:1196] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/3b7acb18668b4c43b2871a8a5aa2eee9
I20260807 18:06:34.613152 16173 ts_tablet_manager.cc:1939] T 3b7acb18668b4c43b2871a8a5aa2eee9 P ce2508eaa73c4d0f95e060d1f5e3aedb: Deleting consensus metadata
I20260807 18:06:34.614805 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.574717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.8.62.189:37593)
0807 18:06:34.575002 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.575012 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.583042 (+ 8030us) server_negotiation.cc:407] Connection header received
0807 18:06:34.583485 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.583494 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.583601 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.583699 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.590595 (+ 6896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.591913 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.597760 (+ 5847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.598577 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.598721 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.604969 (+ 6248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.604977 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.604982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.605033 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.610829 (+ 5796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.611961 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.611986 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.611989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.612209 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.612874 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.612880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.612882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.613344 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.613594 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.614190 (+ 596us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.614447 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:06:34.621500 16172 ts_tablet_manager.cc:1918] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:34.628656 16172 log.cc:1196] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/3b7acb18668b4c43b2871a8a5aa2eee9
I20260807 18:06:34.629755 16172 ts_tablet_manager.cc:1939] T 3b7acb18668b4c43b2871a8a5aa2eee9 P 5948fd37fba0439fb7ff451085ef9d49: Deleting consensus metadata
I20260807 18:06:34.636931 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.580675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.8.62.188:38419)
0807 18:06:34.580891 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.580898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.615750 (+ 34852us) server_negotiation.cc:407] Connection header received
0807 18:06:34.615960 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.615969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.616068 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.616180 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.618236 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.619372 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.621640 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.622663 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.622802 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.627564 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.627574 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.627580 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.627629 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.632944 (+ 5315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.634275 (+ 1331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.634282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.634286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.634498 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.635036 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.635042 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.635046 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.635471 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.635799 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.636205 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.636474 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:06:34.744843 14803 catalog_manager.cc:4841] TS 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865): tablet 3b7acb18668b4c43b2871a8a5aa2eee9 (table test-table [id=9ac2bddc29b84f37b9bba1d1d412b9fe]) successfully deleted
I20260807 18:06:34.745590 14800 catalog_manager.cc:4841] TS 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541): tablet 3b7acb18668b4c43b2871a8a5aa2eee9 (table test-table [id=9ac2bddc29b84f37b9bba1d1d412b9fe]) successfully deleted
I20260807 18:06:34.746376 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 14784
I20260807 18:06:34.800384 16129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.778460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.8.62.189:37593)
0807 18:06:34.778669 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.778677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.778882 (+ 205us) server_negotiation.cc:407] Connection header received
0807 18:06:34.779188 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.779196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.779313 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.779444 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.782723 (+ 3279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.783850 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.786438 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.787189 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.787340 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.792287 (+ 4947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.792296 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.792301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.792365 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.796421 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.797473 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.797479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.797485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.797654 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.798153 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.798162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.798168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.798576 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.798825 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.799715 (+ 890us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.799982 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:06:34.804666 16120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:34.785345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46774 (local address 127.8.62.188:38419)
0807 18:06:34.785606 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:34.785613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:34.785918 (+ 305us) server_negotiation.cc:407] Connection header received
0807 18:06:34.786187 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:34.786194 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:34.786291 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:34.786399 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:34.789540 (+ 3141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.790363 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.792853 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:34.793625 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:34.793787 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:34.797384 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:34.797394 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:34.797398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:34.797449 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:34.801121 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.802014 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.802021 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.802026 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.802249 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:34.802761 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:34.802768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:34.802772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:34.803215 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:34.803511 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:34.803947 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 18:06:34.804230 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:06:35.536450 16183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:35.535183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:39729)
0807 18:06:35.536103 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:35.536266 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":690,"threads_started":1}
W20260807 18:06:35.536926 16114 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:44939 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:35.544704 16184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:35.543666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:49899)
0807 18:06:35.544232 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:35.544393 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":268,"threads_started":1}
W20260807 18:06:35.545193 15816 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:44939 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:35.553455 16185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:35.552393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:50591)
0807 18:06:35.553006 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:35.553183 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":329,"threads_started":1}
W20260807 18:06:35.554020 15968 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:44939 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:36.065220 16186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0af4e45b0e4e4196365f0b3144e996)
I20260807 18:06:36.065448 16187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e0af4e45b0e4e4196365f0b3144e996)
I20260807 18:06:36.065582 16186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } }
I20260807 18:06:36.065842 16187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } }
I20260807 18:06:36.067243 16186 leader_election.cc:288] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939), 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189:37593)
I20260807 18:06:36.067289 16187 leader_election.cc:288] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939), 8d8e31cd37f34c1b89577c57f1c3e014 (127.8.62.188:38419)
I20260807 18:06:36.069185 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.067442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45008)
0807 18:06:36.068690 (+ 1248us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.068878 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":875,"threads_started":1}
I20260807 18:06:36.069540 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.068357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45002)
0807 18:06:36.068999 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.069214 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":319,"threads_started":1}
W20260807 18:06:36.070533 15419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
W20260807 18:06:36.073215 15616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:36.081662 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.076854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45018)
0807 18:06:36.081092 (+ 4238us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.081282 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4086,"thread_start_us":3929,"threads_started":1}
W20260807 18:06:36.082141 15616 leader_election.cc:334] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939): Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:36.083105 16191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.069135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.8.62.189:37593)
0807 18:06:36.070956 (+ 1821us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.070968 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.070991 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:06:36.071085 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.071096 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.071236 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.072241 (+ 1005us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:36.072629 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.074751 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.078841 (+ 4090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.081459 (+ 2618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.082304 (+ 845us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.082418 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.082476 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.082660 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1707,"thread_start_us":1505,"threads_started":1}
I20260807 18:06:36.083701 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.069891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:37593 (local address 127.0.0.1:39978)
0807 18:06:36.070252 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.070272 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:36.070407 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:36.071491 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:36.071500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:36.071546 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:36.072027 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.072048 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.075060 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.075071 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.078560 (+ 3489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.078586 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.081706 (+ 3120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.081714 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.081971 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.082017 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:06:36.082100 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 18:06:36.083409 (+ 1309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":1080}
I20260807 18:06:36.084652 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.083795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45030)
0807 18:06:36.084103 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.084363 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 18:06:36.084607 15444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" is_pre_election: true
I20260807 18:06:36.084857 16192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.070432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.8.62.188:38419)
0807 18:06:36.073376 (+ 2944us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.073384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.073407 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:06:36.073632 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.073638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.073720 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.073860 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:36.075038 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.077914 (+ 2876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.081769 (+ 3855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.083464 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.083679 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.083800 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.084370 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.084506 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":987,"server-negotiator.queue_time_us":2798,"thread_start_us":2402,"threads_started":1}
W20260807 18:06:36.084892 15419 leader_election.cc:334] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939): Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:36.085071 15444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d8e31cd37f34c1b89577c57f1c3e014 in term 1.
I20260807 18:06:36.085345 16188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.068798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:38419 (local address 127.0.0.1:46790)
0807 18:06:36.069412 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.069429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:36.069602 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:36.074273 (+ 4671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:36.074283 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:36.074329 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:36.074731 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.074749 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.078157 (+ 3408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.078180 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.081466 (+ 3286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.081492 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.083743 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.083956 (+ 213us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.084186 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.084222 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:06:36.084328 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 18:06:36.084503 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":44,"thread_start_us":1102,"threads_started":1}
I20260807 18:06:36.085683 15616 leader_election.cc:302] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ea4e7b4593a4c3d80fe664c2fffb6fb, 8d8e31cd37f34c1b89577c57f1c3e014; no voters: 9e0af4e45b0e4e4196365f0b3144e996
I20260807 18:06:36.086012 16186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 18:06:36.086141 16186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9e0af4e45b0e4e4196365f0b3144e996)
I20260807 18:06:36.086258 16186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:06:36.086606 15642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" is_pre_election: true
I20260807 18:06:36.088656 16186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } }
I20260807 18:06:36.089148 15642 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ea4e7b4593a4c3d80fe664c2fffb6fb in current term 2: Already voted for candidate 8d8e31cd37f34c1b89577c57f1c3e014 in this term.
I20260807 18:06:36.089365 16186 leader_election.cc:288] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 election: Requested vote from peers 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939), 5ea4e7b4593a4c3d80fe664c2fffb6fb (127.8.62.189:37593)
I20260807 18:06:36.089879 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.089459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45044)
0807 18:06:36.089620 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.089722 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 18:06:36.090058 15444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb"
I20260807 18:06:36.090355 15444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:06:36.090400 15419 leader_election.cc:302] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ea4e7b4593a4c3d80fe664c2fffb6fb; no voters: 8d8e31cd37f34c1b89577c57f1c3e014, 9e0af4e45b0e4e4196365f0b3144e996
I20260807 18:06:36.091138 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.090706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45046)
0807 18:06:36.090899 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.090999 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260807 18:06:36.091325 15616 leader_election.cc:334] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939): Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
I20260807 18:06:36.093089 15444 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d8e31cd37f34c1b89577c57f1c3e014 in term 2.
I20260807 18:06:36.093324 16187 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 18:06:36.093593 15616 leader_election.cc:302] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ea4e7b4593a4c3d80fe664c2fffb6fb, 8d8e31cd37f34c1b89577c57f1c3e014; no voters: 9e0af4e45b0e4e4196365f0b3144e996
I20260807 18:06:36.093919 16186 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:06:36.094506 16186 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [term 2 LEADER]: Becoming Leader. State: Replica: 8d8e31cd37f34c1b89577c57f1c3e014, State: Running, Role: LEADER
I20260807 18:06:36.095057 16186 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } }
I20260807 18:06:36.097385 16196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d8e31cd37f34c1b89577c57f1c3e014. Latest consensus state: current_term: 2 leader_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:36.097965 16196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:36.098773 16198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:36.102993 16198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:36.104046 16198 catalog_manager.cc:1259] Loaded cluster ID: b09efc591e7749d8a2c7f618beea1679
I20260807 18:06:36.104121 16198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:36.105461 16198 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:36.106364 16198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Loaded TSK: 0
I20260807 18:06:36.107333 16198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:06:36.144871 15631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:36.152766 15631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:36.157871 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.157414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45064)
0807 18:06:36.157618 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.157716 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:06:36.157961 15444 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [term 2 FOLLOWER]: Refusing update from remote peer 8d8e31cd37f34c1b89577c57f1c3e014: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 18:06:36.159122 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.158618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45080)
0807 18:06:36.158834 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.158930 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260807 18:06:36.159296 15616 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 -> Peer 9e0af4e45b0e4e4196365f0b3144e996 (127.8.62.190:44939): Couldn't send request to peer 9e0af4e45b0e4e4196365f0b3144e996. Status: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 18:06:36.159405 16196 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 18:06:36.162653 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d8e31cd37f34c1b89577c57f1c3e014. Latest consensus state: current_term: 2 leader_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:36.162931 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:36.165134 16196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:36.165558 16196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:36.167066 16187 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "9e0af4e45b0e4e4196365f0b3144e996" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44939 } } peers { permanent_uuid: "5ea4e7b4593a4c3d80fe664c2fffb6fb" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 37593 } attrs { promote: false } } peers { permanent_uuid: "8d8e31cd37f34c1b89577c57f1c3e014" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 38419 } attrs { promote: false } } }
I20260807 18:06:36.167328 16187 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ea4e7b4593a4c3d80fe664c2fffb6fb [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:06:36.200381 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.180680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.8.62.129:41749)
0807 18:06:36.192422 (+ 11742us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.192432 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.192463 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:06:36.192582 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.192590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.192706 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.192851 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:36.194096 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.195252 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.198046 (+ 2794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.199300 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.199441 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.199564 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.199873 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.200028 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11539,"thread_start_us":11762,"threads_started":1}
I20260807 18:06:36.200381 16195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.180290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37742)
0807 18:06:36.180456 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.180478 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:36.180623 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:36.192879 (+ 12256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:36.192884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:36.192919 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:36.193292 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.193305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.195459 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.195464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.197796 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.197813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.199521 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.199527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.199697 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.199724 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:06:36.199843 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:06:36.200013 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":3}
I20260807 18:06:36.201326 16189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.178879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:39541 (local address 127.0.0.1:52924)
0807 18:06:36.179090 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.179111 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:36.179242 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:36.193299 (+ 14057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:36.193303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:36.193329 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:36.193519 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.193530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.195155 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.195164 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.197426 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.197446 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.200461 (+ 3015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.200468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.200704 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.200748 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:06:36.200814 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:06:36.200985 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":5}
I20260807 18:06:36.201754 15749 tablet_service.cc:1467] Processing CreateTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 (DEFAULT_TABLE table=test-table [id=273d67ef238e481781f9ee21f2602b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:36.201851 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.181290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:33865 (local address 127.0.0.1:52878)
0807 18:06:36.194002 (+ 12712us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.194025 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:36.194156 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:36.194760 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:36.194765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:36.194791 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:36.195259 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.195276 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.196855 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.196861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.199144 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:36.199160 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.200988 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.200994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:36.201190 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.201221 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:06:36.201321 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:06:36.201474 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12578,"mutex_wait_us":45,"thread_start_us":12398,"threads_started":1}
I20260807 18:06:36.202232 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.180906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52878 (local address 127.8.62.131:33865)
0807 18:06:36.193435 (+ 12529us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.193444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.194285 (+ 841us) server_negotiation.cc:407] Connection header received
0807 18:06:36.194371 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.194378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.194453 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.194548 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:36.195480 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.196673 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.199324 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.200640 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.200771 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.201663 (+ 892us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.201726 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.201896 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12360,"thread_start_us":12238,"threads_started":1}
I20260807 18:06:36.202337 15749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c54ae9ea4742bb9b98831b5d3dd475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:36.203073 16046 tablet_service.cc:1467] Processing CreateTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 (DEFAULT_TABLE table=test-table [id=273d67ef238e481781f9ee21f2602b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:36.203528 16046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c54ae9ea4742bb9b98831b5d3dd475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:36.203685 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.179361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52924 (local address 127.8.62.130:39541)
0807 18:06:36.192904 (+ 13543us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.192910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.192928 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:06:36.193007 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.193023 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.193113 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.193683 (+ 570us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:36.193840 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.194974 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.198806 (+ 3832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.200231 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.203100 (+ 2869us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.203198 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.203251 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.203377 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13418,"thread_start_us":14534,"threads_started":1}
I20260807 18:06:36.204635 15900 tablet_service.cc:1467] Processing CreateTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 (DEFAULT_TABLE table=test-table [id=273d67ef238e481781f9ee21f2602b26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:36.205138 15900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41c54ae9ea4742bb9b98831b5d3dd475. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:36.210057 16209 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: Bootstrap starting.
I20260807 18:06:36.211534 16210 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Bootstrap starting.
I20260807 18:06:36.212344 16211 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Bootstrap starting.
I20260807 18:06:36.212987 16209 tablet_bootstrap.cc:654] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:36.213696 16210 tablet_bootstrap.cc:654] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:36.214356 16211 tablet_bootstrap.cc:654] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:36.215920 16209 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: No bootstrap required, opened a new log
I20260807 18:06:36.216115 16209 ts_tablet_manager.cc:1397] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:06:36.217070 16210 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: No bootstrap required, opened a new log
I20260807 18:06:36.217339 16210 ts_tablet_manager.cc:1397] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 18:06:36.217353 16209 raft_consensus.cc:348] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.217720 16209 raft_consensus.cc:374] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:36.217855 16209 raft_consensus.cc:729] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2508eaa73c4d0f95e060d1f5e3aedb, State: Initialized, Role: FOLLOWER
I20260807 18:06:36.218335 16209 consensus_queue.cc:260] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.218607 16210 raft_consensus.cc:348] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.219097 16210 raft_consensus.cc:374] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:36.219245 16210 raft_consensus.cc:729] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5948fd37fba0439fb7ff451085ef9d49, State: Initialized, Role: FOLLOWER
I20260807 18:06:36.219650 16210 consensus_queue.cc:260] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.223443 16211 tablet_bootstrap.cc:492] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: No bootstrap required, opened a new log
I20260807 18:06:36.223690 16211 ts_tablet_manager.cc:1397] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 18:06:36.224849 16211 raft_consensus.cc:348] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.225200 16211 raft_consensus.cc:374] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:36.225323 16211 raft_consensus.cc:729] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055a3f0da5a74e959525874718da97bd, State: Initialized, Role: FOLLOWER
I20260807 18:06:36.225704 16211 consensus_queue.cc:260] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.226827 16209 ts_tablet_manager.cc:1428] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260807 18:06:36.228788 16210 ts_tablet_manager.cc:1428] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260807 18:06:36.233496 16211 ts_tablet_manager.cc:1428] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Time spent starting tablet: real 0.010s user 0.003s sys 0.002s
I20260807 18:06:36.234876 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.230049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:33017)
0807 18:06:36.234344 (+ 4295us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.234548 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4128,"threads_started":1}
I20260807 18:06:36.236384 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.227494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:49185)
0807 18:06:36.235904 (+ 8410us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.236092 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8243,"thread_start_us":8067,"threads_started":1}
I20260807 18:06:36.238499 15815 heartbeater.cc:503] Master 127.8.62.188:38419 was elected leader, sending a full tablet report...
I20260807 18:06:36.240087 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.238566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:36653)
0807 18:06:36.239844 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.239941 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1141}
I20260807 18:06:36.240401 16113 heartbeater.cc:503] Master 127.8.62.188:38419 was elected leader, sending a full tablet report...
I20260807 18:06:36.241871 16216 raft_consensus.cc:482] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:36.242097 16216 raft_consensus.cc:504] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.242600 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.241120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:56839)
0807 18:06:36.242340 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.242436 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1103}
I20260807 18:06:36.242902 16216 leader_election.cc:288] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541), ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:36.243533 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.235220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:34541)
0807 18:06:36.241500 (+ 6280us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.241688 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6121,"thread_start_us":5943,"threads_started":1}
I20260807 18:06:36.243860 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c54ae9ea4742bb9b98831b5d3dd475" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055a3f0da5a74e959525874718da97bd" is_pre_election: true
I20260807 18:06:36.243537 15967 heartbeater.cc:503] Master 127.8.62.188:38419 was elected leader, sending a full tablet report...
I20260807 18:06:36.244132 15920 raft_consensus.cc:2454] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 0.
I20260807 18:06:36.244225 15769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c54ae9ea4742bb9b98831b5d3dd475" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" is_pre_election: true
I20260807 18:06:36.244504 15769 raft_consensus.cc:2454] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 0.
I20260807 18:06:36.244678 15998 leader_election.cc:302] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 055a3f0da5a74e959525874718da97bd, 5948fd37fba0439fb7ff451085ef9d49; no voters:
I20260807 18:06:36.245139 16216 raft_consensus.cc:2781] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:36.245286 16216 raft_consensus.cc:482] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:36.245385 16216 raft_consensus.cc:3037] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:36.246663 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.246255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:34939)
0807 18:06:36.246416 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.246506 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:06:36.248334 16216 raft_consensus.cc:504] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.249037 16216 leader_election.cc:288] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 election: Requested vote from peers 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541), ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749)
I20260807 18:06:36.249651 15769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c54ae9ea4742bb9b98831b5d3dd475" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb"
I20260807 18:06:36.249748 15920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41c54ae9ea4742bb9b98831b5d3dd475" candidate_uuid: "5948fd37fba0439fb7ff451085ef9d49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "055a3f0da5a74e959525874718da97bd"
I20260807 18:06:36.249893 15769 raft_consensus.cc:3037] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:36.250023 15920 raft_consensus.cc:3037] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:36.252414 15769 raft_consensus.cc:2454] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 1.
I20260807 18:06:36.252645 15920 raft_consensus.cc:2454] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5948fd37fba0439fb7ff451085ef9d49 in term 1.
I20260807 18:06:36.252975 16001 leader_election.cc:302] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5948fd37fba0439fb7ff451085ef9d49, ce2508eaa73c4d0f95e060d1f5e3aedb; no voters:
I20260807 18:06:36.253341 16216 raft_consensus.cc:2781] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:36.253839 16216 raft_consensus.cc:686] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 LEADER]: Becoming Leader. State: Replica: 5948fd37fba0439fb7ff451085ef9d49, State: Running, Role: LEADER
I20260807 18:06:36.254307 16216 consensus_queue.cc:237] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } }
I20260807 18:06:36.255040 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.254478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:41877)
0807 18:06:36.254674 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.254796 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 18:06:36.257624 15631 catalog_manager.cc:5495] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "5948fd37fba0439fb7ff451085ef9d49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5948fd37fba0439fb7ff451085ef9d49" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 33865 } health_report { overall_health: HEALTHY } } }
I20260807 18:06:36.289791 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.283942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.8.62.131:33865)
0807 18:06:36.284215 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.284220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.284287 (+ 67us) server_negotiation.cc:407] Connection header received
0807 18:06:36.284616 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.284627 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.284702 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.284787 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:36.285223 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.286027 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.287663 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.288124 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.288397 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.288831 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:36.288954 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:36.289027 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.289241 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.289318 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 18:06:36.293445 15920 raft_consensus.cc:1261] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Refusing update from remote peer 5948fd37fba0439fb7ff451085ef9d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:36.293478 15769 raft_consensus.cc:1261] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Refusing update from remote peer 5948fd37fba0439fb7ff451085ef9d49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:36.294211 16223 consensus_queue.cc:1035] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "055a3f0da5a74e959525874718da97bd" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 39541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:36.294588 16216 consensus_queue.cc:1035] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce2508eaa73c4d0f95e060d1f5e3aedb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:36.297745 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.297168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:54935)
0807 18:06:36.297366 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.297524 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:06:36.299117 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.297566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:54491)
0807 18:06:36.298748 (+ 1182us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.298895 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1038}
I20260807 18:06:36.303150 16226 mvcc.cc:205] Tried to move back new op lower bound from 7315972080810803200 to 7315972080660168704. Current Snapshot: MvccSnapshot[applied={T|T < 7315972080810803200}]
I20260807 18:06:36.304901 16228 mvcc.cc:205] Tried to move back new op lower bound from 7315972080810803200 to 7315972080660168704. Current Snapshot: MvccSnapshot[applied={T|T < 7315972080810803200}]
I20260807 18:06:36.305223 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.304730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.129:45111)
0807 18:06:36.304930 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.305041 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:06:36.306993 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.306396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.131:51211)
0807 18:06:36.306659 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.306779 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260807 18:06:36.307267 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.306436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.8.62.130:38449)
0807 18:06:36.306930 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.307080 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260807 18:06:36.314777 15631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:36.315063 15631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:36.320967 15631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d8e31cd37f34c1b89577c57f1c3e014: Sending DeleteTablet for 3 replicas of tablet 41c54ae9ea4742bb9b98831b5d3dd475
I20260807 18:06:36.322243 15900 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:36 UTC) from {username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:52924
I20260807 18:06:36.322297 15749 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:36 UTC) from {username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:37742
I20260807 18:06:36.322309 16046 tablet_service.cc:1514] Processing DeleteTablet for tablet 41c54ae9ea4742bb9b98831b5d3dd475 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:36 UTC) from {username='oryx', principal='oryx/127.8.62.188@KRBTEST.COM'} at 127.0.0.1:52878
I20260807 18:06:36.323982 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15614
I20260807 18:06:36.333354 16238 tablet_replica.cc:330] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: stopping tablet replica
I20260807 18:06:36.333604 16238 raft_consensus.cc:2229] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:36.334107 16238 raft_consensus.cc:2258] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:36.335342 16238 ts_tablet_manager.cc:1905] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:36.341760 16236 tablet_replica.cc:330] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: stopping tablet replica
I20260807 18:06:36.342041 16236 raft_consensus.cc:2229] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:36.342425 16236 raft_consensus.cc:2258] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:36.343091 16237 tablet_replica.cc:330] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: stopping tablet replica
I20260807 18:06:36.343299 16237 raft_consensus.cc:2229] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:36.343628 16237 raft_consensus.cc:2258] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:36.345237 16237 ts_tablet_manager.cc:1905] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:36.345842 16238 ts_tablet_manager.cc:1918] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:36.346017 16238 log.cc:1196] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/41c54ae9ea4742bb9b98831b5d3dd475
I20260807 18:06:36.346518 16238 ts_tablet_manager.cc:1939] T 41c54ae9ea4742bb9b98831b5d3dd475 P 5948fd37fba0439fb7ff451085ef9d49: Deleting consensus metadata
I20260807 18:06:36.348044 15618 catalog_manager.cc:4841] TS 5948fd37fba0439fb7ff451085ef9d49 (127.8.62.131:33865): tablet 41c54ae9ea4742bb9b98831b5d3dd475 (table test-table [id=273d67ef238e481781f9ee21f2602b26]) successfully deleted
I20260807 18:06:36.348420 16236 ts_tablet_manager.cc:1905] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:36.349699 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.325407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.8.62.129:41749)
0807 18:06:36.325691 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.325700 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.327105 (+ 1405us) server_negotiation.cc:407] Connection header received
0807 18:06:36.327457 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.327466 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.327578 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.327681 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:36.330484 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.332022 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.334405 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.335125 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.335231 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.339219 (+ 3988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:36.339246 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:36.339270 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:36.339325 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:36.345003 (+ 5678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.346287 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.346326 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.346333 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.346573 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.347103 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.347108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.347112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.347701 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:36.348174 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.348960 (+ 786us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.349239 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:06:36.351471 15810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:36.355490 16236 ts_tablet_manager.cc:1918] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:36.355654 16236 log.cc:1196] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/41c54ae9ea4742bb9b98831b5d3dd475
I20260807 18:06:36.356211 16236 ts_tablet_manager.cc:1939] T 41c54ae9ea4742bb9b98831b5d3dd475 P 055a3f0da5a74e959525874718da97bd: Deleting consensus metadata
I20260807 18:06:36.357524 15615 catalog_manager.cc:4841] TS 055a3f0da5a74e959525874718da97bd (127.8.62.130:39541): tablet 41c54ae9ea4742bb9b98831b5d3dd475 (table test-table [id=273d67ef238e481781f9ee21f2602b26]) successfully deleted
I20260807 18:06:36.453472 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15614
I20260807 18:06:36.453472 16237 ts_tablet_manager.cc:1918] T 41c54ae9ea4742bb9b98831b5d3dd475 P ce2508eaa73c4d0f95e060d1f5e3aedb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:36.473354 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15828
I20260807 18:06:36.473873 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.473346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37760)
0807 18:06:36.473561 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.473671 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:06:36.474175 15618 catalog_manager.cc:4584] TS ce2508eaa73c4d0f95e060d1f5e3aedb (127.8.62.129:41749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41c54ae9ea4742bb9b98831b5d3dd475: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
I20260807 18:06:36.487490 16205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.474328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52930 (local address 127.8.62.130:39541)
0807 18:06:36.474565 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.474570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.474598 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:06:36.474706 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.474711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.474782 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.474897 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:36.476211 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.477153 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.478421 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.478854 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.478998 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.481678 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:36.481695 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:36.481711 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:36.481747 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:36.484659 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.485388 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.485397 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.485400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.485545 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.485933 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.485939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.485942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.486309 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:36.486627 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.486914 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.487162 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:06:36.488201 15962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:36.535154 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.534628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37774)
0807 18:06:36.534880 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.534983 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 18:06:36.536343 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.535987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37790)
0807 18:06:36.536144 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.536222 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260807 18:06:36.585801 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15828
I20260807 18:06:36.605158 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.604701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37802)
0807 18:06:36.604898 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.604986 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:06:36.606475 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.605915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37814)
0807 18:06:36.606293 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.606359 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
I20260807 18:06:36.606710 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15981
I20260807 18:06:36.620755 16207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.607501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.8.62.131:33865)
0807 18:06:36.607670 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:36.607675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:36.607693 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:06:36.607779 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:36.607783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:36.607854 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:36.607943 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:36.609727 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.610539 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.611824 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:36.612318 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:36.612415 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:36.615083 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:36.615109 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:36.615126 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:36.615161 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:36.618087 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.618806 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.618813 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.618816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.619021 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:36.619353 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:36.619357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:36.619359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:36.619724 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:36.619949 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:36.620214 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:06:36.620402 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:06:36.621526 16108 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:36.651676 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.651097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45084)
0807 18:06:36.651385 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.651494 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:06:36.652796 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.652393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:44939 (local address 127.0.0.1:45090)
0807 18:06:36.652611 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.652686 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:44939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 18:06:36.708321 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.707797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37816)
0807 18:06:36.708055 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.708155 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260807 18:06:36.709451 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.709082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37822)
0807 18:06:36.709251 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.709343 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 18:06:36.748409 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15981
I20260807 18:06:36.774943 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15290
I20260807 18:06:36.776218 15465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:36.881634 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.881030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37828)
0807 18:06:36.881316 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.881436 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 18:06:36.882874 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:36.882451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41749 (local address 127.0.0.1:37844)
0807 18:06:36.882646 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:36.882731 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:41749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 18:06:36.898563 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15290
I20260807 18:06:36.921569 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 15391
I20260807 18:06:36.922837 15662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:37.046118 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 15391
2026-08-07T18:06:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10348 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:37 dist-test-slave-rnrw krb5kdc[16247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:37 dist-test-slave-rnrw krb5kdc[16247](info): set up 2 sockets
Aug 07 18:06:37 dist-test-slave-rnrw krb5kdc[16247](info): commencing operation
krb5kdc: starting...
W20260807 18:06:38.185217 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.009s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:38 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:38Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab.
Entry for principal oryx/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab.
I20260807 18:06:38.440716 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35425
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab
--principal=oryx/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:38.799392 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:38.799726 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:38.799899 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:38.807353 16260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:38.807490 16260 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 18:06:38.807572 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:38.807677 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:38.807767 16260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:38.807864 16260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:38.818131 16260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35425
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.190.keytab
--principal=oryx/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:38.819547 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:38.820953 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:38.833710 16265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:38.833730 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:38.833773 16266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:38.834702 16260 server_base.cc:1034] running on GCE node
I20260807 18:06:38.835572 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:38.837285 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:38.838573 16260 hybrid_clock.cc:648] HybridClock initialized: now 1786125998838506 us; error 39 us; skew 500 ppm
I20260807 18:06:38.838958 16260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:38 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125998, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:38.842725 16260 init.cc:377] Logged in from keytab as oryx/127.8.62.190@KRBTEST.COM (short username oryx)
I20260807 18:06:38.844679 16260 webserver.cc:460] Webserver started at http://127.8.62.190:46509/ using document root <none> and password file <none>
I20260807 18:06:38.845525 16260 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:38.845671 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:38.846029 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:38.850003 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "ef6379433061441ab2fdc0151302d424"
format_stamp: "Formatted at 2026-08-07 18:06:38 on dist-test-slave-rnrw"
server_key: "8b43011a4f7854fa576ffde9780c2796"
server_key_iv: "f17a6170423f12ed856d2121b8f07218"
server_key_version: "encryptionkey@0"
I20260807 18:06:38.850979 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "ef6379433061441ab2fdc0151302d424"
format_stamp: "Formatted at 2026-08-07 18:06:38 on dist-test-slave-rnrw"
server_key: "8b43011a4f7854fa576ffde9780c2796"
server_key_iv: "f17a6170423f12ed856d2121b8f07218"
server_key_version: "encryptionkey@0"
I20260807 18:06:38.856756 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:06:38.860723 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:38.862358 16260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:38.862679 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "ef6379433061441ab2fdc0151302d424"
format_stamp: "Formatted at 2026-08-07 18:06:38 on dist-test-slave-rnrw"
server_key: "8b43011a4f7854fa576ffde9780c2796"
server_key_iv: "f17a6170423f12ed856d2121b8f07218"
server_key_version: "encryptionkey@0"
I20260807 18:06:38.862948 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:38.877314 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:38.882256 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:38.882567 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:38.907384 16260 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:35425
I20260807 18:06:38.907399 16340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:35425 every 8 connection(s)
I20260807 18:06:38.909049 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:38.913873 16341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:38.916411 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 16260
I20260807 18:06:38.916695 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:38.917583 8442 external_mini_cluster.cc:1428] Setting key a1692b3065527ed07d45d7c352260dbc
I20260807 18:06:38.927816 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Bootstrap starting.
I20260807 18:06:38.931967 16341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:38.933434 16341 log.cc:824] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:06:38 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:38.936827 16341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: No bootstrap required, opened a new log
I20260807 18:06:38.943552 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:38.920075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.8.62.190:35425)
0807 18:06:38.921204 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:38.921217 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:38.921264 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:06:38.923224 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:38.923251 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:38.923734 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:38.924139 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:38.926172 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:38.927523 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:38.929355 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:38.930147 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:38.930311 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:38.935584 (+ 5273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:38.935594 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:38.935610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:38.935652 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:38.939003 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:38.939919 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:38.939926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:38.939932 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:38.940148 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:38.940678 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:38.940681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:38.940683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:38.941274 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:38.941513 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:38.941982 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 18:06:38.942440 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":879,"thread_start_us":585,"threads_started":1}
I20260807 18:06:38.946234 16341 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } }
I20260807 18:06:38.946732 16341 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:38.946846 16341 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef6379433061441ab2fdc0151302d424, State: Initialized, Role: FOLLOWER
I20260807 18:06:38.947507 16341 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } }
I20260807 18:06:38.947754 16341 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:38.947898 16341 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:38.948079 16341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:38.950596 16341 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } }
I20260807 18:06:38.951156 16341 leader_election.cc:302] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef6379433061441ab2fdc0151302d424; no voters:
I20260807 18:06:38.951762 16341 leader_election.cc:288] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:38.951974 16346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:38.952761 16346 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [term 1 LEADER]: Becoming Leader. State: Replica: ef6379433061441ab2fdc0151302d424, State: Running, Role: LEADER
I20260807 18:06:38.953336 16346 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } }
I20260807 18:06:38.953935 16341 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:38.956328 16347 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef6379433061441ab2fdc0151302d424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } } }
I20260807 18:06:38.957121 16347 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:38.956687 16348 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef6379433061441ab2fdc0151302d424. Latest consensus state: current_term: 1 leader_uuid: "ef6379433061441ab2fdc0151302d424" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef6379433061441ab2fdc0151302d424" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35425 } } }
I20260807 18:06:38.957856 16348 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:38.958117 16355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:38.963369 16355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:38.972718 16355 catalog_manager.cc:1347] Generated new cluster ID: 06ff306c8a41480485728ce5ca5fd97e
I20260807 18:06:38.972898 16355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:38.983511 16355 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:38.985486 16355 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:38.995025 16355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Generated new TSK 0
I20260807 18:06:38.996304 16355 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
I20260807 18:06:39.060997 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35425
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:39.422019 16368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:39.422338 16368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:39.422703 16368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:39.430433 16368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:39.430562 16368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:39.430714 16368 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:06:39.430848 16368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:39.441002 16368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:39.442473 16368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:39.444113 16368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:39.459051 16374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:39.459134 16377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:39.459043 16375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:39.462280 16368 server_base.cc:1034] running on GCE node
I20260807 18:06:39.463029 16368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:39.464224 16368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:39.465509 16368 hybrid_clock.cc:648] HybridClock initialized: now 1786125999465421 us; error 57 us; skew 500 ppm
I20260807 18:06:39.465867 16368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:39 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125999, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:39.469494 16368 init.cc:377] Logged in from keytab as oryx/127.8.62.129@KRBTEST.COM (short username oryx)
I20260807 18:06:39.471143 16368 webserver.cc:460] Webserver started at http://127.8.62.129:36431/ using document root <none> and password file <none>
I20260807 18:06:39.471946 16368 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:39.472092 16368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:39.472424 16368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:39.475713 16368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "b1807e3d1a654e4d9a03607bf5b71d10"
format_stamp: "Formatted at 2026-08-07 18:06:39 on dist-test-slave-rnrw"
server_key: "003e2ecfe93bbb7558d1f5e4bf9d981f"
server_key_iv: "03670c4e2e65d7db785d6cf9e7fecc31"
server_key_version: "encryptionkey@0"
I20260807 18:06:39.476747 16368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "b1807e3d1a654e4d9a03607bf5b71d10"
format_stamp: "Formatted at 2026-08-07 18:06:39 on dist-test-slave-rnrw"
server_key: "003e2ecfe93bbb7558d1f5e4bf9d981f"
server_key_iv: "03670c4e2e65d7db785d6cf9e7fecc31"
server_key_version: "encryptionkey@0"
I20260807 18:06:39.482947 16368 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:06:39.487349 16384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:39.489147 16368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:39.489377 16368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "b1807e3d1a654e4d9a03607bf5b71d10"
format_stamp: "Formatted at 2026-08-07 18:06:39 on dist-test-slave-rnrw"
server_key: "003e2ecfe93bbb7558d1f5e4bf9d981f"
server_key_iv: "03670c4e2e65d7db785d6cf9e7fecc31"
server_key_version: "encryptionkey@0"
I20260807 18:06:39.489660 16368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:39.514595 16368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:39.520124 16368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:39.520483 16368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:39.523692 16368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:39.523825 16368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:39.524008 16368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:39.524101 16368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:06:39 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125999, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:39.554524 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:39.524398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:39039 (local address 127.8.62.190:35425)
0807 18:06:39.525171 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:39.525178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:39.526580 (+ 1402us) server_negotiation.cc:407] Connection header received
0807 18:06:39.529937 (+ 3357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:39.529944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:39.530050 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:39.530205 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:39.533085 (+ 2880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.534544 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.538539 (+ 3995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.539369 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.539562 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:39.545723 (+ 6161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:39.545733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:39.545740 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:39.545797 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:39.549928 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:39.551581 (+ 1653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:39.551590 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:39.551596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:39.551799 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:39.552412 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:39.552419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:39.552422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:39.552816 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:39.552996 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:39.553843 (+ 847us) server_negotiation.cc:299] Negotiation successful
0807 18:06:39.554114 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":468,"threads_started":1}
I20260807 18:06:39.556021 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:39.525312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.129:39039)
0807 18:06:39.526337 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:39.526398 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:39.529436 (+ 3038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:39.530679 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:39.530699 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:39.531655 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:39.532817 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:39.532866 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.534765 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.534775 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:39.538264 (+ 3489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:39.538300 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.539744 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.539756 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:39.540242 (+ 486us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:39.541385 (+ 1143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:39.541436 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:39.545374 (+ 3938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:39.550175 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:39.550184 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:39.550202 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:39.551324 (+ 1122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:39.551951 (+ 627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:39.551962 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:39.551966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:39.552254 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:39.552966 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:39.552975 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:39.553420 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 18:06:39.553719 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 18:06:39.554321 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":379,"threads_started":1}
I20260807 18:06:39.576457 16368 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:42689
I20260807 18:06:39.576504 16508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:42689 every 8 connection(s)
I20260807 18:06:39.577867 16368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:39.580276 16509 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35425
I20260807 18:06:39.580570 16509 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:39.581421 16509 heartbeater.cc:511] Master 127.8.62.190:35425 requested a full tablet report, sending...
I20260807 18:06:39.582819 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:39.563004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.129:39907)
0807 18:06:39.565087 (+ 2083us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:39.565109 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:39.565307 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:39.565779 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:39.565788 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:39.566729 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:39.567190 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:39.567206 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.568910 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.568919 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:39.571553 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:39.571579 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.572868 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.572880 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:39.573144 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:39.574350 (+ 1206us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:39.574374 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:39.575710 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:39.579775 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:39.579782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:39.579786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:39.580364 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:39.580934 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:39.580939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:39.580942 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:39.581107 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:39.581769 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:39.581774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:39.581976 (+ 202us) client_negotiation.cc:769] Sending connection context
0807 18:06:39.582139 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 18:06:39.582451 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1920,"thread_start_us":1767,"threads_started":1}
I20260807 18:06:39.582916 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:39.563085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:39907 (local address 127.8.62.190:35425)
0807 18:06:39.563321 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:39.563331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:39.565225 (+ 1894us) server_negotiation.cc:407] Connection header received
0807 18:06:39.565457 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:39.565465 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:39.565571 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:39.565686 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:39.567459 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.568685 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.571771 (+ 3086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:39.572589 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:39.572768 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:39.576004 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:39.576014 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:39.576019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:39.576072 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:39.579592 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:39.580620 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:39.580628 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:39.580632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:39.580796 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:39.581234 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:39.581238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:39.581240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:39.581607 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:39.581772 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:39.582258 (+ 486us) server_negotiation.cc:299] Negotiation successful
0807 18:06:39.582518 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:06:39.583803 16292 ts_manager.cc:194] Registered new tserver with Master: b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689)
I20260807 18:06:39.585513 16292 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:39039
I20260807 18:06:39.586848 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 16368
I20260807 18:06:39.587055 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:39.587905 8442 external_mini_cluster.cc:1428] Setting key 2a1404e5c311915f72fbdfce95b7b235
WARNING: no policy specified for oryx/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
I20260807 18:06:39.644995 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35425
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:39.997898 16514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:39.998224 16514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:39.998390 16514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:40.006114 16514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:40.006258 16514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:40.006410 16514 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:06:40.006508 16514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:40.017009 16514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:40.018522 16514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:40.019968 16514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:40.034277 16520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:40.034332 16519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:40.034386 16522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:40.035924 16514 server_base.cc:1034] running on GCE node
I20260807 18:06:40.036722 16514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:40.037825 16514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:40.039088 16514 hybrid_clock.cc:648] HybridClock initialized: now 1786126000039025 us; error 35 us; skew 500 ppm
I20260807 18:06:40.039458 16514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:40.043149 16514 init.cc:377] Logged in from keytab as oryx/127.8.62.130@KRBTEST.COM (short username oryx)
I20260807 18:06:40.044916 16514 webserver.cc:460] Webserver started at http://127.8.62.130:39071/ using document root <none> and password file <none>
I20260807 18:06:40.045707 16514 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:40.045850 16514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:40.046212 16514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:40.049643 16514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "5c4326d6cf6546208d17176ab5c0c939"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "e4d553d393b5362d130bfa66e298520e"
server_key_iv: "dbe4adce5672e6be34e625ac6998f824"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.050520 16514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "5c4326d6cf6546208d17176ab5c0c939"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "e4d553d393b5362d130bfa66e298520e"
server_key_iv: "dbe4adce5672e6be34e625ac6998f824"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.056457 16514 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 18:06:40.060606 16530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:40.062104 16514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:40.062295 16514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "5c4326d6cf6546208d17176ab5c0c939"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "e4d553d393b5362d130bfa66e298520e"
server_key_iv: "dbe4adce5672e6be34e625ac6998f824"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.062546 16514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:40.079051 16514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:40.084095 16514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:40.084336 16514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:40.087270 16514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:40.087358 16514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:40.087581 16514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:40.087652 16514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:40.112253 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.089238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:46019 (local address 127.8.62.190:35425)
0807 18:06:40.089763 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.089772 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.089810 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:06:40.092412 (+ 2602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.092418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.092518 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.092671 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.095371 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.096856 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.098706 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.099551 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.099737 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.105196 (+ 5459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:40.105203 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:40.105206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:40.105249 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:40.108185 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.109364 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.109376 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.109382 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.109620 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.110208 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.110213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.110216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.110560 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:40.110754 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.111590 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.111861 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":1081,"threads_started":1}
I20260807 18:06:40.113916 16541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.088478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.130:46019)
0807 18:06:40.089561 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.089620 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.092077 (+ 2457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.093015 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.093026 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.093933 (+ 907us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:40.095036 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.095075 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.097067 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.097071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.098521 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.098536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.099893 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.099903 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.100177 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.101035 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:40.101060 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:40.104854 (+ 3794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:40.108390 (+ 3536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.108401 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.108422 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.109160 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.109789 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.109793 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.109795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.110046 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.110713 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:40.110722 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:40.111163 (+ 441us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.111461 (+ 298us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.112078 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":680,"threads_started":1}
I20260807 18:06:40.140218 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.121929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:43889 (local address 127.8.62.190:35425)
0807 18:06:40.122179 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.122189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.122866 (+ 677us) server_negotiation.cc:407] Connection header received
0807 18:06:40.123033 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.123042 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.123157 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.123294 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.124404 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.125773 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.127907 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.128461 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.128650 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.131783 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:40.131789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:40.131792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:40.131829 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:40.136062 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.137236 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.137245 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.137252 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.137463 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.138129 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.138136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.138140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.138607 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:40.138782 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.139477 (+ 695us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.139798 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:06:40.140414 16595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.121759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.130:43889)
0807 18:06:40.122723 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.122745 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.122891 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.123375 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.123378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.123918 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:40.124234 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.124244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.125997 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.126005 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.127725 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.127739 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.128732 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.128742 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.129052 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.129928 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:40.129962 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:40.131472 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:40.136272 (+ 4800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.136284 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.136291 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.137060 (+ 769us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.137656 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.137668 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.137674 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.137904 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.138795 (+ 891us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:40.138803 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:40.139099 (+ 296us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.139352 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.139887 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":628,"threads_started":1}
I20260807 18:06:40.149892 16514 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:40397
I20260807 18:06:40.149911 16655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:40397 every 8 connection(s)
I20260807 18:06:40.151319 16514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:40.153630 16656 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35425
I20260807 18:06:40.153941 16656 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:40.154736 16656 heartbeater.cc:511] Master 127.8.62.190:35425 requested a full tablet report, sending...
I20260807 18:06:40.156378 16292 ts_manager.cc:194] Registered new tserver with Master: 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:40.157374 16292 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:46019
I20260807 18:06:40.161401 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 16514
I20260807 18:06:40.161549 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:40.161966 8442 external_mini_cluster.cc:1428] Setting key ceff79f9b99f1c073921d04cc8b27824
WARNING: no policy specified for oryx/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
I20260807 18:06:40.218215 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35425
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:40661
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:40.577999 16660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:40.578414 16660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:40.578613 16660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:40.587222 16660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:40.587419 16660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:40.587602 16660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 18:06:40.587747 16660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:40.589354 16509 heartbeater.cc:503] Master 127.8.62.190:35425 was elected leader, sending a full tablet report...
I20260807 18:06:40.599056 16660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40661
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.62.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:40.600405 16660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:40.601855 16660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:40.615971 16665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:40.616060 16668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:40.616086 16666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:40.617203 16660 server_base.cc:1034] running on GCE node
I20260807 18:06:40.617956 16660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:40.619024 16660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:40.620280 16660 hybrid_clock.cc:648] HybridClock initialized: now 1786126000620191 us; error 72 us; skew 500 ppm
I20260807 18:06:40.620656 16660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:40.624033 16660 init.cc:377] Logged in from keytab as oryx/127.8.62.131@KRBTEST.COM (short username oryx)
I20260807 18:06:40.625586 16660 webserver.cc:460] Webserver started at http://127.8.62.131:38411/ using document root <none> and password file <none>
I20260807 18:06:40.626349 16660 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:40.626490 16660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:40.626834 16660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:40.630184 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "87693dfd86f05a74bf25d3fd29bda9f2"
server_key_iv: "269d813f44157f0b250955c2816160ce"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.631063 16660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "87693dfd86f05a74bf25d3fd29bda9f2"
server_key_iv: "269d813f44157f0b250955c2816160ce"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.636783 16660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:06:40.641021 16675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:40.642709 16660 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:40.642939 16660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
format_stamp: "Formatted at 2026-08-07 18:06:40 on dist-test-slave-rnrw"
server_key: "87693dfd86f05a74bf25d3fd29bda9f2"
server_key_iv: "269d813f44157f0b250955c2816160ce"
server_key_version: "encryptionkey@0"
I20260807 18:06:40.643198 16660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:40.672672 16660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:40.677634 16660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:40.677940 16660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:40.680812 16660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:40.680928 16660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:40.681104 16660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:40.681195 16660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:40.707710 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.681771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:42351 (local address 127.8.62.190:35425)
0807 18:06:40.682331 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.682339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.683901 (+ 1562us) server_negotiation.cc:407] Connection header received
0807 18:06:40.686013 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.686019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.686128 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.686254 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.689620 (+ 3366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.690683 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.693418 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.694122 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.694271 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.699958 (+ 5687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:40.699963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:40.699967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:40.700041 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:40.703765 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.705102 (+ 1337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.705109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.705113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.705268 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.705757 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.705763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.705766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.706079 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:40.706247 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.707050 (+ 803us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.707293 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":265,"threads_started":1}
I20260807 18:06:40.708920 16686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.682466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.131:42351)
0807 18:06:40.683451 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.683509 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.685664 (+ 2155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.686771 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.686788 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.688083 (+ 1295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:40.689330 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.689373 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.690866 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.690881 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.693142 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.693168 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.694381 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.694389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.694837 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.695872 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:40.695896 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:40.699638 (+ 3742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:40.704037 (+ 4399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.704052 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.704078 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.704928 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.705394 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.705400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.705403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.705622 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.706230 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:40.706239 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:40.706647 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.706933 (+ 286us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.707542 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":431,"threads_started":1}
I20260807 18:06:40.735599 16800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:32881 every 8 connection(s)
I20260807 18:06:40.735605 16660 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:32881
I20260807 18:06:40.737502 16660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:40.740298 16801 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35425
I20260807 18:06:40.740615 16801 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:40.741496 16801 heartbeater.cc:511] Master 127.8.62.190:35425 requested a full tablet report, sending...
I20260807 18:06:40.741793 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.724153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.8.62.131:41973)
0807 18:06:40.725004 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.725025 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.725202 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.725579 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.725584 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.726508 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:40.727149 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.727166 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.728620 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.728625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.730250 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.730263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.731409 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.731416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.731600 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.732365 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:40.732385 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:40.733535 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:40.738121 (+ 4586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.738130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.738136 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.739257 (+ 1121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.739806 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:40.739811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:40.739814 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:40.739986 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:40.740762 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:40.740767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:40.740952 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.741071 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.741335 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":488,"threads_started":1}
I20260807 18:06:40.741856 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.724051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41973 (local address 127.8.62.190:35425)
0807 18:06:40.724336 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.724344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.725132 (+ 788us) server_negotiation.cc:407] Connection header received
0807 18:06:40.725336 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.725340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.725420 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.725508 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.727346 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.728392 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.730411 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.731208 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.731358 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.733826 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:40.733837 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:40.733844 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:40.733891 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:40.737873 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.739413 (+ 1540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.739422 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.739427 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.739665 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.740141 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.740145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.740147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.740623 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:40.740796 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.741226 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.741501 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:06:40.743037 16291 ts_manager.cc:194] Registered new tserver with Master: 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:40.744019 16291 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:42351
I20260807 18:06:40.744423 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 16660
I20260807 18:06:40.744624 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:40.745078 8442 external_mini_cluster.cc:1428] Setting key ad4317d7acda705e950ff9d7039783d8
I20260807 18:06:40.748699 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:06:40.807850 16685 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:40.795915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.8.62.190:35425)
0807 18:06:40.796144 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.796148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.796441 (+ 293us) server_negotiation.cc:407] Connection header received
0807 18:06:40.796671 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.796677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.796746 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.796848 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.798406 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.799193 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.800613 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.801066 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.801180 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.807488 (+ 6308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33902: BlockingRecv error: recv got EOF from 127.0.0.1:33902 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 18:06:40 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126000, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:40.826658 16685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.811712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.8.62.190:35425)
0807 18:06:40.811986 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.811993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.812237 (+ 244us) server_negotiation.cc:407] Connection header received
0807 18:06:40.812379 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.812383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.812609 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.812709 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:40.814021 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.814745 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.816083 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.816693 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.816900 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.821043 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:40.821049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:40.821053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:40.821092 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:40.823939 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.824777 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.824782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.824784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.824914 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:40.825304 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:40.825308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:40.825311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:40.825610 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:40.825819 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.826192 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.826382 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 18:06:40.831059 16291 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:40.838193 16291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:40.887096 16819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.851831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:32881 (local address 127.0.0.1:52274)
0807 18:06:40.869096 (+ 17265us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.869160 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.869655 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.872153 (+ 2498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.872168 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.872259 (+ 91us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:40.873239 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.873270 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.877783 (+ 4513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.877804 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.882118 (+ 4314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.882140 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.884894 (+ 2754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.884912 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.885358 (+ 446us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.885620 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.885885 (+ 265us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.886213 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15515,"mutex_wait_us":72,"thread_start_us":15961,"threads_started":1}
I20260807 18:06:40.888892 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.852175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.8.62.131:32881)
0807 18:06:40.867051 (+ 14876us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.867084 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.869418 (+ 2334us) server_negotiation.cc:407] Connection header received
0807 18:06:40.870146 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.870171 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.870688 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.871111 (+ 423us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:40.873759 (+ 2648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.877439 (+ 3680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.882419 (+ 4980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.884297 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.884623 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.885094 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.887844 (+ 2750us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.888184 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13897,"thread_start_us":14095,"threads_started":1}
I20260807 18:06:40.894033 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.854138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.8.62.130:40397)
0807 18:06:40.876717 (+ 22579us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.876736 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.878635 (+ 1899us) server_negotiation.cc:407] Connection header received
0807 18:06:40.878954 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.878968 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.879447 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.879812 (+ 365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:40.881633 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.883933 (+ 2300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.888667 (+ 4734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.891224 (+ 2557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.891479 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.891933 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.893116 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.893445 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19089,"thread_start_us":18853,"threads_started":1}
I20260807 18:06:40.894522 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.852401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.8.62.129:42689)
0807 18:06:40.877858 (+ 25457us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:40.877883 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:40.877965 (+ 82us) server_negotiation.cc:407] Connection header received
0807 18:06:40.878215 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:40.878236 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:40.878720 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:40.880829 (+ 2109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:40.881247 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.883890 (+ 2643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.889422 (+ 5532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.891974 (+ 2552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.892275 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.892759 (+ 484us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:40.893641 (+ 882us) server_negotiation.cc:299] Negotiation successful
0807 18:06:40.893918 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24924,"thread_start_us":24636,"threads_started":1}
I20260807 18:06:40.895735 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.853891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40397 (local address 127.0.0.1:58704)
0807 18:06:40.878393 (+ 24502us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.878435 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.878688 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.880185 (+ 1497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.880195 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.880238 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:40.881140 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.881186 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.884230 (+ 3044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.884248 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.888280 (+ 4032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.888309 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.891601 (+ 3292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.891620 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.892065 (+ 445us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.892126 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.892332 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.892685 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24221,"mutex_wait_us":8,"thread_start_us":13723,"threads_started":1}
I20260807 18:06:40.895907 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:40.852093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.0.0.1:40658)
0807 18:06:40.868307 (+ 16214us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:40.868407 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:40.868837 (+ 430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:40.879171 (+ 10334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:40.879178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:40.879245 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:40.880284 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.880316 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.884257 (+ 3941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.884322 (+ 65us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.889026 (+ 4704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:40.889048 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:40.892355 (+ 3307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:40.892375 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:40.892827 (+ 452us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:40.892955 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 18:06:40.893117 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 18:06:40.894975 (+ 1858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15774,"mutex_wait_us":70,"thread_start_us":25705,"threads_started":1}
I20260807 18:06:40.896416 16728 tablet_service.cc:1467] Processing CreateTablet for tablet f14a7d3a44684647be2400dc1551f49e (DEFAULT_TABLE table=test-table [id=5f1292a2617b498cb9ff753929167aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:40.899371 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a7d3a44684647be2400dc1551f49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:40.901896 16436 tablet_service.cc:1467] Processing CreateTablet for tablet f14a7d3a44684647be2400dc1551f49e (DEFAULT_TABLE table=test-table [id=5f1292a2617b498cb9ff753929167aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:40.902411 16583 tablet_service.cc:1467] Processing CreateTablet for tablet f14a7d3a44684647be2400dc1551f49e (DEFAULT_TABLE table=test-table [id=5f1292a2617b498cb9ff753929167aab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:40.905055 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a7d3a44684647be2400dc1551f49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:40.905670 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f14a7d3a44684647be2400dc1551f49e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:40.935192 16825 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Bootstrap starting.
I20260807 18:06:40.946305 16826 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Bootstrap starting.
I20260807 18:06:40.948797 16827 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Bootstrap starting.
I20260807 18:06:40.953753 16827 tablet_bootstrap.cc:654] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:40.953732 16826 tablet_bootstrap.cc:654] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:40.954618 16825 tablet_bootstrap.cc:654] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:40.956111 16826 log.cc:824] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:40.956188 16827 log.cc:824] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:40.956851 16825 log.cc:824] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:40.967150 16825 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: No bootstrap required, opened a new log
I20260807 18:06:40.967974 16825 ts_tablet_manager.cc:1397] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent bootstrapping tablet: real 0.033s user 0.020s sys 0.000s
I20260807 18:06:40.968329 16827 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: No bootstrap required, opened a new log
I20260807 18:06:40.968955 16827 ts_tablet_manager.cc:1397] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.004s
I20260807 18:06:40.974098 16826 tablet_bootstrap.cc:492] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: No bootstrap required, opened a new log
I20260807 18:06:40.974622 16826 ts_tablet_manager.cc:1397] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Time spent bootstrapping tablet: real 0.029s user 0.013s sys 0.012s
I20260807 18:06:40.982025 16825 raft_consensus.cc:348] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:40.982955 16825 raft_consensus.cc:374] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:40.983155 16825 raft_consensus.cc:729] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Initialized, Role: FOLLOWER
I20260807 18:06:40.984192 16825 consensus_queue.cc:260] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:40.989933 16827 raft_consensus.cc:348] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:40.990878 16827 raft_consensus.cc:374] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:40.991081 16827 raft_consensus.cc:729] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4326d6cf6546208d17176ab5c0c939, State: Initialized, Role: FOLLOWER
I20260807 18:06:40.991710 16801 heartbeater.cc:503] Master 127.8.62.190:35425 was elected leader, sending a full tablet report...
I20260807 18:06:40.992914 16825 ts_tablet_manager.cc:1428] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent starting tablet: real 0.025s user 0.017s sys 0.007s
I20260807 18:06:40.992631 16827 consensus_queue.cc:260] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:40.995394 16656 heartbeater.cc:503] Master 127.8.62.190:35425 was elected leader, sending a full tablet report...
I20260807 18:06:40.995679 16826 raft_consensus.cc:348] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:40.996723 16826 raft_consensus.cc:374] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:40.996922 16827 ts_tablet_manager.cc:1428] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Time spent starting tablet: real 0.028s user 0.019s sys 0.008s
I20260807 18:06:40.996970 16826 raft_consensus.cc:729] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Initialized, Role: FOLLOWER
I20260807 18:06:40.998306 16826 consensus_queue.cc:260] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:41.001284 16826 ts_tablet_manager.cc:1428] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Time spent starting tablet: real 0.026s user 0.027s sys 0.000s
I20260807 18:06:41.071709 16833 raft_consensus.cc:482] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:41.072149 16833 raft_consensus.cc:504] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:41.075439 16833 leader_election.cc:288] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
W20260807 18:06:41.081557 16510 tablet.cc:2369] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:41.089248 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.076601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:50659 (local address 127.8.62.130:40397)
0807 18:06:41.077008 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.077017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.078241 (+ 1224us) server_negotiation.cc:407] Connection header received
0807 18:06:41.078621 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.078630 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.078782 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.078930 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:41.080420 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.082270 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.085918 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.087282 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.087628 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.087852 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.088368 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.088660 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142}
I20260807 18:06:41.089275 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.077018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:46319 (local address 127.8.62.131:32881)
0807 18:06:41.077337 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.077355 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.078241 (+ 886us) server_negotiation.cc:407] Connection header received
0807 18:06:41.078622 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.078629 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.078782 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.078930 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:41.080420 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.082250 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.086038 (+ 3788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.087311 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.087743 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.087903 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.088431 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.088715 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:06:41.089618 16835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.075858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40397 (local address 127.8.62.129:50659)
0807 18:06:41.077976 (+ 2118us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:41.078033 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:41.078410 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:41.078985 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:41.078990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:41.079089 (+ 99us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:41.080127 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.080140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.082595 (+ 2455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.082599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.085590 (+ 2991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.085621 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.087756 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.087777 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.088084 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.088146 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 18:06:41.088277 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 18:06:41.088687 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1836,"mutex_wait_us":93,"spinlock_wait_cycles":105344,"thread_start_us":1443,"threads_started":1}
I20260807 18:06:41.090168 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.075878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:32881 (local address 127.8.62.129:46319)
0807 18:06:41.077979 (+ 2101us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:41.078050 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:41.078333 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:41.078985 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:41.078990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:41.079091 (+ 101us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:41.080068 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.080127 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.082558 (+ 2431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.082567 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.085713 (+ 3146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.085773 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.087841 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.087850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.088114 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.088167 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 18:06:41.088352 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 18:06:41.088687 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1918,"mutex_wait_us":304,"spinlock_wait_cycles":37376,"thread_start_us":964,"threads_started":1}
I20260807 18:06:41.091464 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a7d3a44684647be2400dc1551f49e" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939" is_pre_election: true
I20260807 18:06:41.092037 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a7d3a44684647be2400dc1551f49e" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" is_pre_election: true
I20260807 18:06:41.092411 16608 raft_consensus.cc:2454] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.092907 16748 raft_consensus.cc:2454] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.093766 16387 leader_election.cc:302] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.094766 16833 raft_consensus.cc:2781] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:41.095085 16833 raft_consensus.cc:482] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:41.095309 16833 raft_consensus.cc:3037] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.099225 16833 raft_consensus.cc:504] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:41.100371 16833 leader_election.cc:288] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Requested vote from peers 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:41.101315 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a7d3a44684647be2400dc1551f49e" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
I20260807 18:06:41.101449 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f14a7d3a44684647be2400dc1551f49e" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939"
I20260807 18:06:41.101791 16748 raft_consensus.cc:3037] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.101825 16608 raft_consensus.cc:3037] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.105803 16608 raft_consensus.cc:2454] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.106619 16387 leader_election.cc:302] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.106678 16748 raft_consensus.cc:2454] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.107157 16833 raft_consensus.cc:2781] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:41.108132 16833 raft_consensus.cc:686] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 LEADER]: Becoming Leader. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Running, Role: LEADER
I20260807 18:06:41.109272 16833 consensus_queue.cc:237] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } }
I20260807 18:06:41.117806 16290 catalog_manager.cc:5495] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } health_report { overall_health: UNKNOWN } } }
W20260807 18:06:41.153874 16657 tablet.cc:2369] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:41.203243 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.194924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.8.62.129:42689)
0807 18:06:41.195192 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.195199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.195386 (+ 187us) server_negotiation.cc:407] Connection header received
0807 18:06:41.195700 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.195705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.195860 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.196065 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:41.196682 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.197717 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.199878 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.200519 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.200760 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.201458 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:41.202413 (+ 955us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:41.202591 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.202793 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.202906 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:06:41.217360 16748 raft_consensus.cc:1261] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.217396 16608 raft_consensus.cc:1261] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.219035 16833 consensus_queue.cc:1035] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.219578 16836 consensus_queue.cc:1035] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.240383 16839 mvcc.cc:205] Tried to move back new op lower bound from 7315972100969766912 to 7315972100551544832. Current Snapshot: MvccSnapshot[applied={T|T < 7315972100969766912}]
W20260807 18:06:41.241138 16802 tablet.cc:2369] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:06:41.244041 16840 mvcc.cc:205] Tried to move back new op lower bound from 7315972100969766912 to 7315972100551544832. Current Snapshot: MvccSnapshot[applied={T|T < 7315972100969766912}]
I20260807 18:06:41.271816 16292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:41.272259 16292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:41.279017 16292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Sending DeleteTablet for 3 replicas of tablet f14a7d3a44684647be2400dc1551f49e
I20260807 18:06:41.281299 16728 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a7d3a44684647be2400dc1551f49e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52274
I20260807 18:06:41.282039 16436 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a7d3a44684647be2400dc1551f49e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40658
I20260807 18:06:41.283095 16583 tablet_service.cc:1514] Processing DeleteTablet for tablet f14a7d3a44684647be2400dc1551f49e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58704
I20260807 18:06:41.287361 16292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:41.290145 16292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:41.297034 16849 tablet_replica.cc:330] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: stopping tablet replica
I20260807 18:06:41.301901 16851 tablet_replica.cc:330] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: stopping tablet replica
I20260807 18:06:41.301836 16849 raft_consensus.cc:2229] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:41.302882 16849 raft_consensus.cc:2258] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:41.303169 16851 raft_consensus.cc:2229] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:41.305711 16850 tablet_replica.cc:330] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: stopping tablet replica
I20260807 18:06:41.306555 16850 raft_consensus.cc:2229] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:41.308089 16850 raft_consensus.cc:2258] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:41.308424 16849 ts_tablet_manager.cc:1905] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:41.309894 16851 raft_consensus.cc:2258] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:41.313431 16436 tablet_service.cc:1467] Processing CreateTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d (DEFAULT_TABLE table=test-table [id=c6cd609ed31346858da5607caee8bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:41.314653 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2568c7c47fc5484fbf3c4cd655a3e75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.316031 16728 tablet_service.cc:1467] Processing CreateTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d (DEFAULT_TABLE table=test-table [id=c6cd609ed31346858da5607caee8bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:41.316849 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2568c7c47fc5484fbf3c4cd655a3e75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.321494 16583 tablet_service.cc:1467] Processing CreateTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d (DEFAULT_TABLE table=test-table [id=c6cd609ed31346858da5607caee8bce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:41.322408 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2568c7c47fc5484fbf3c4cd655a3e75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.322836 16849 ts_tablet_manager.cc:1918] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:41.323155 16849 log.cc:1196] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/f14a7d3a44684647be2400dc1551f49e
I20260807 18:06:41.324208 16849 ts_tablet_manager.cc:1939] T f14a7d3a44684647be2400dc1551f49e P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting consensus metadata
I20260807 18:06:41.327695 16851 ts_tablet_manager.cc:1905] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:41.328122 16276 catalog_manager.cc:4841] TS 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881): tablet f14a7d3a44684647be2400dc1551f49e (table test-table [id=5f1292a2617b498cb9ff753929167aab]) successfully deleted
I20260807 18:06:41.331460 16850 ts_tablet_manager.cc:1905] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:41.336863 16825 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Bootstrap starting.
I20260807 18:06:41.340044 16825 tablet_bootstrap.cc:654] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.342913 16827 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Bootstrap starting.
I20260807 18:06:41.346151 16827 tablet_bootstrap.cc:654] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.356395 16850 ts_tablet_manager.cc:1918] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:41.356863 16850 log.cc:1196] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/f14a7d3a44684647be2400dc1551f49e
I20260807 18:06:41.358006 16826 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Bootstrap starting.
I20260807 18:06:41.358831 16851 ts_tablet_manager.cc:1918] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:41.359272 16851 log.cc:1196] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/f14a7d3a44684647be2400dc1551f49e
I20260807 18:06:41.359784 16825 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: No bootstrap required, opened a new log
I20260807 18:06:41.360103 16825 ts_tablet_manager.cc:1397] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.003s
I20260807 18:06:41.360710 16851 ts_tablet_manager.cc:1939] T f14a7d3a44684647be2400dc1551f49e P 5c4326d6cf6546208d17176ab5c0c939: Deleting consensus metadata
I20260807 18:06:41.361478 16826 tablet_bootstrap.cc:654] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.362005 16825 raft_consensus.cc:348] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.362627 16825 raft_consensus.cc:374] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.362856 16825 raft_consensus.cc:729] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.363441 16825 consensus_queue.cc:260] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.358040 16850 ts_tablet_manager.cc:1939] T f14a7d3a44684647be2400dc1551f49e P b1807e3d1a654e4d9a03607bf5b71d10: Deleting consensus metadata
I20260807 18:06:41.366199 16278 catalog_manager.cc:4841] TS 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397): tablet f14a7d3a44684647be2400dc1551f49e (table test-table [id=5f1292a2617b498cb9ff753929167aab]) successfully deleted
I20260807 18:06:41.367015 16825 ts_tablet_manager.cc:1428] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260807 18:06:41.368217 16827 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: No bootstrap required, opened a new log
I20260807 18:06:41.368536 16827 ts_tablet_manager.cc:1397] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.009s
I20260807 18:06:41.369957 16277 catalog_manager.cc:4841] TS b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689): tablet f14a7d3a44684647be2400dc1551f49e (table test-table [id=5f1292a2617b498cb9ff753929167aab]) successfully deleted
I20260807 18:06:41.370487 16827 raft_consensus.cc:348] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.370924 16827 raft_consensus.cc:374] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.371138 16827 raft_consensus.cc:729] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4326d6cf6546208d17176ab5c0c939, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.371662 16827 consensus_queue.cc:260] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.372501 16826 tablet_bootstrap.cc:492] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: No bootstrap required, opened a new log
I20260807 18:06:41.372830 16826 ts_tablet_manager.cc:1397] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260807 18:06:41.374572 16827 ts_tablet_manager.cc:1428] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260807 18:06:41.374552 16826 raft_consensus.cc:348] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.374984 16826 raft_consensus.cc:374] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.375237 16826 raft_consensus.cc:729] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.375780 16826 consensus_queue.cc:260] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.377112 16826 ts_tablet_manager.cc:1428] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260807 18:06:41.437395 16833 raft_consensus.cc:482] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:41.437790 16833 raft_consensus.cc:504] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.438972 16833 leader_election.cc:288] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397), 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:41.439975 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2568c7c47fc5484fbf3c4cd655a3e75d" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" is_pre_election: true
I20260807 18:06:41.440027 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2568c7c47fc5484fbf3c4cd655a3e75d" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939" is_pre_election: true
I20260807 18:06:41.440472 16748 raft_consensus.cc:2454] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.440586 16608 raft_consensus.cc:2454] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.441360 16385 leader_election.cc:302] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.441915 16833 raft_consensus.cc:2781] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:41.442114 16833 raft_consensus.cc:482] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:41.442270 16833 raft_consensus.cc:3037] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.445948 16833 raft_consensus.cc:504] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.447677 16833 leader_election.cc:288] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Requested vote from peers 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397), 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:41.448879 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2568c7c47fc5484fbf3c4cd655a3e75d" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939"
I20260807 18:06:41.449220 16608 raft_consensus.cc:3037] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.449061 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2568c7c47fc5484fbf3c4cd655a3e75d" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
I20260807 18:06:41.449488 16748 raft_consensus.cc:3037] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.452536 16608 raft_consensus.cc:2454] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.453583 16387 leader_election.cc:302] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.454248 16833 raft_consensus.cc:2781] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:41.454535 16833 raft_consensus.cc:686] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 LEADER]: Becoming Leader. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Running, Role: LEADER
I20260807 18:06:41.454632 16748 raft_consensus.cc:2454] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.455950 16833 consensus_queue.cc:237] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.461839 16291 catalog_manager.cc:5495] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
Aug 07 18:06:41 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786125998, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.190@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:41.554433 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.520889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35425 (local address 127.0.0.1:33928)
0807 18:06:41.522132 (+ 1243us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:41.522179 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:41.522513 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:41.523168 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:41.523172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:41.524431 (+ 1259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:41.524827 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.524846 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.526514 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.526532 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.528208 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.528233 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.529607 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.529626 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.530134 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.536827 (+ 6693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:41.536876 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:41.541906 (+ 5030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:41.548326 (+ 6420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:41.548364 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:41.548404 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:41.549796 (+ 1392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:41.550670 (+ 874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:41.550677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:41.550693 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:41.551056 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:41.552206 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:41.552231 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:41.552705 (+ 474us) client_negotiation.cc:769] Sending connection context
0807 18:06:41.553066 (+ 361us) client_negotiation.cc:240] Negotiation successful
0807 18:06:41.553702 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":41,"thread_start_us":651,"threads_started":1}
I20260807 18:06:41.554566 16863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.520959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33928 (local address 127.8.62.190:35425)
0807 18:06:41.522741 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.522751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.522780 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:06:41.522849 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.522853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.522946 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.523149 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:41.525103 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.526277 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.528467 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.529269 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.529578 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.542405 (+ 12827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:41.542428 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:41.542432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:41.542515 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:41.548054 (+ 5539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:41.550130 (+ 2076us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:41.550135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:41.550146 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:41.550404 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:41.551312 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:41.551318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:41.551320 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:41.551944 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:41.552186 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.553278 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.553851 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":411,"server-negotiator.queue_time_us":1563,"thread_start_us":1259,"threads_started":1}
I20260807 18:06:41.565104 16291 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:33928:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 18:06:41.567384 16291 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:41.582198 16436 tablet_service.cc:1467] Processing CreateTablet for tablet 58ae09a96bb34cafbd9fe623ebb44775 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=850923bd0ff14956927a8aa535e8a2d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:06:41.583794 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ae09a96bb34cafbd9fe623ebb44775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.585587 16728 tablet_service.cc:1467] Processing CreateTablet for tablet 58ae09a96bb34cafbd9fe623ebb44775 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=850923bd0ff14956927a8aa535e8a2d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:06:41.585556 16583 tablet_service.cc:1467] Processing CreateTablet for tablet 58ae09a96bb34cafbd9fe623ebb44775 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=850923bd0ff14956927a8aa535e8a2d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 18:06:41.586752 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ae09a96bb34cafbd9fe623ebb44775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.586766 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ae09a96bb34cafbd9fe623ebb44775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:41.601508 16826 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: Bootstrap starting.
I20260807 18:06:41.605979 16826 tablet_bootstrap.cc:654] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.616786 16826 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: No bootstrap required, opened a new log
I20260807 18:06:41.617018 16827 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: Bootstrap starting.
I20260807 18:06:41.617200 16826 ts_tablet_manager.cc:1397] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.007s
I20260807 18:06:41.618871 16825 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: Bootstrap starting.
I20260807 18:06:41.618829 16826 raft_consensus.cc:348] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.619336 16826 raft_consensus.cc:374] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.619524 16826 raft_consensus.cc:729] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.620492 16827 tablet_bootstrap.cc:654] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.620190 16826 consensus_queue.cc:260] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.621465 16826 ts_tablet_manager.cc:1428] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:41.621189 16833 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.621795 16833 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.623783 16825 tablet_bootstrap.cc:654] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:41.631203 16825 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: No bootstrap required, opened a new log
I20260807 18:06:41.631521 16825 ts_tablet_manager.cc:1397] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260807 18:06:41.635509 16827 tablet_bootstrap.cc:492] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: No bootstrap required, opened a new log
I20260807 18:06:41.635838 16827 ts_tablet_manager.cc:1397] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.000s
I20260807 18:06:41.637266 16827 raft_consensus.cc:348] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.637719 16827 raft_consensus.cc:374] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.637862 16827 raft_consensus.cc:729] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4326d6cf6546208d17176ab5c0c939, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.637566 16825 raft_consensus.cc:348] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.638394 16825 raft_consensus.cc:374] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:41.638612 16825 raft_consensus.cc:729] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Initialized, Role: FOLLOWER
I20260807 18:06:41.638367 16827 consensus_queue.cc:260] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.639237 16825 consensus_queue.cc:260] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.639216 16832 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.639796 16832 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.640323 16831 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.640937 16831 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.643738 16825 ts_tablet_manager.cc:1428] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent starting tablet: real 0.012s user 0.003s sys 0.001s
I20260807 18:06:41.646085 16827 ts_tablet_manager.cc:1428] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260807 18:06:41.660331 16833 raft_consensus.cc:482] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:41.660727 16833 raft_consensus.cc:504] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.662088 16833 leader_election.cc:288] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397), 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:41.662948 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ae09a96bb34cafbd9fe623ebb44775" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939" is_pre_election: true
I20260807 18:06:41.663363 16608 raft_consensus.cc:2454] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.663589 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ae09a96bb34cafbd9fe623ebb44775" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" is_pre_election: true
I20260807 18:06:41.664016 16387 leader_election.cc:302] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.664004 16748 raft_consensus.cc:2454] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 0.
I20260807 18:06:41.664515 16833 raft_consensus.cc:2781] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:41.664731 16833 raft_consensus.cc:482] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:41.664945 16833 raft_consensus.cc:3037] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.669185 16833 raft_consensus.cc:504] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.670579 16833 leader_election.cc:288] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Requested vote from peers 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397), 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:41.671277 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ae09a96bb34cafbd9fe623ebb44775" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939"
I20260807 18:06:41.671490 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ae09a96bb34cafbd9fe623ebb44775" candidate_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6"
I20260807 18:06:41.671775 16608 raft_consensus.cc:3037] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.671897 16748 raft_consensus.cc:3037] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:41.676507 16608 raft_consensus.cc:2454] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.677088 16748 raft_consensus.cc:2454] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1807e3d1a654e4d9a03607bf5b71d10 in term 1.
I20260807 18:06:41.677521 16387 leader_election.cc:302] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:41.678166 16833 raft_consensus.cc:2781] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:41.678531 16833 raft_consensus.cc:686] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 LEADER]: Becoming Leader. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Running, Role: LEADER
I20260807 18:06:41.679070 16833 consensus_queue.cc:237] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:41.682109 16836 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: TxnStatusTablet state changed. Reason: New leader b1807e3d1a654e4d9a03607bf5b71d10. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.682601 16836 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:06:41.685187 16291 catalog_manager.cc:5495] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20260807 18:06:41.687388 16868 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 18:06:41.687940 16868 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 18:06:41.719719 16862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.708022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.0.0.1:40680)
0807 18:06:41.708764 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:41.708818 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:41.709122 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:41.710048 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:41.710053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:41.710074 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:06:41.710364 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.710384 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.712867 (+ 2483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.712881 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.714533 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.714570 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.716763 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.716771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.717501 (+ 730us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.717692 (+ 191us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:06:41.718642 (+ 950us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:06:41.718694 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 18:06:41.718835 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 18:06:41.719154 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 18:06:41.719755 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.707909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40680 (local address 127.8.62.129:42689)
0807 18:06:41.709303 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.709333 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.709372 (+ 39us) server_negotiation.cc:407] Connection header received
0807 18:06:41.709516 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.709523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.709653 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.709953 (+ 300us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:41.710613 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.712615 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.714862 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.716313 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.716581 (+ 268us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.718074 (+ 1493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:41.718407 (+ 333us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:41.718625 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.718914 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.719108 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":879,"threads_started":1}
I20260807 18:06:41.726035 16748 raft_consensus.cc:1261] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.726154 16608 raft_consensus.cc:1261] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.727268 16836 consensus_queue.cc:1035] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.727795 16833 consensus_queue.cc:1035] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.737953 16831 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: TxnStatusTablet state changed. Reason: New leader b1807e3d1a654e4d9a03607bf5b71d10. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.738857 16831 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.744874 16832 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: TxnStatusTablet state changed. Reason: New leader b1807e3d1a654e4d9a03607bf5b71d10. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.745471 16832 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.746636 16844 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.747247 16844 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:06:41.750530 16836 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.751127 16836 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P b1807e3d1a654e4d9a03607bf5b71d10: This TxnStatusTablet replica's current role is: LEADER
I20260807 18:06:41.752943 16831 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.753513 16831 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 146d5c2a01e54432a6f1bd4a6de18fc6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.773066 16832 tablet_replica.cc:439] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } } }
I20260807 18:06:41.773648 16832 tablet_replica.cc:442] T 58ae09a96bb34cafbd9fe623ebb44775 P 5c4326d6cf6546208d17176ab5c0c939: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 18:06:41.799367 16870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.790284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:50393 (local address 127.8.62.129:42689)
0807 18:06:41.790754 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:41.790777 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:41.791960 (+ 1183us) server_negotiation.cc:407] Connection header received
0807 18:06:41.792349 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:41.792353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:41.792432 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:41.792639 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:41.793690 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.795220 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.797469 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.797913 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.798065 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.798514 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:41.798629 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:41.798729 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:41.798859 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 18:06:41.798969 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":77}
I20260807 18:06:41.799620 16876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:41.790397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.8.62.129:50393)
0807 18:06:41.791779 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:41.791809 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:41.792115 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:41.792884 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:41.792887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:41.792896 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:06:41.793380 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.793397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.795470 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.795477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.797312 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:41.797327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:41.798094 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:41.798101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:41.798267 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:41.798326 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:06:41.798728 (+ 402us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:06:41.798740 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:06:41.798840 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 18:06:41.799042 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":42,"thread_start_us":786,"threads_started":1}
I20260807 18:06:41.822381 16608 raft_consensus.cc:1261] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.822984 16748 raft_consensus.cc:1261] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Refusing update from remote peer b1807e3d1a654e4d9a03607bf5b71d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:41.824254 16833 consensus_queue.cc:1035] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.825455 16833 consensus_queue.cc:1035] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:41.977133 16290 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:41.977757 16290 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:41.984115 16290 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Sending DeleteTablet for 3 replicas of tablet 2568c7c47fc5484fbf3c4cd655a3e75d
I20260807 18:06:41.986267 16436 tablet_service.cc:1514] Processing DeleteTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40658
I20260807 18:06:41.986778 16728 tablet_service.cc:1514] Processing DeleteTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52274
I20260807 18:06:41.987452 16583 tablet_service.cc:1514] Processing DeleteTablet for tablet 2568c7c47fc5484fbf3c4cd655a3e75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:41 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58704
I20260807 18:06:42.001021 16882 tablet_replica.cc:330] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: stopping tablet replica
I20260807 18:06:42.001150 16884 tablet_replica.cc:330] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: stopping tablet replica
I20260807 18:06:42.001546 16882 raft_consensus.cc:2229] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:42.001916 16884 raft_consensus.cc:2229] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:42.002600 16884 raft_consensus.cc:2258] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.002595 16882 raft_consensus.cc:2258] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.004194 16883 tablet_replica.cc:330] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: stopping tablet replica
I20260807 18:06:42.004647 16883 raft_consensus.cc:2229] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:42.004752 16882 ts_tablet_manager.cc:1905] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:42.005084 16884 ts_tablet_manager.cc:1905] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:42.005111 16883 raft_consensus.cc:2258] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.010499 16883 ts_tablet_manager.cc:1905] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:42.021485 16882 ts_tablet_manager.cc:1918] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:42.021811 16882 log.cc:1196] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/2568c7c47fc5484fbf3c4cd655a3e75d
I20260807 18:06:42.022712 16882 ts_tablet_manager.cc:1939] T 2568c7c47fc5484fbf3c4cd655a3e75d P b1807e3d1a654e4d9a03607bf5b71d10: Deleting consensus metadata
I20260807 18:06:42.025048 16277 catalog_manager.cc:4841] TS b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689): tablet 2568c7c47fc5484fbf3c4cd655a3e75d (table test-table [id=c6cd609ed31346858da5607caee8bce2]) successfully deleted
I20260807 18:06:42.030490 16883 ts_tablet_manager.cc:1918] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:42.030758 16883 log.cc:1196] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/2568c7c47fc5484fbf3c4cd655a3e75d
I20260807 18:06:42.031617 16883 ts_tablet_manager.cc:1939] T 2568c7c47fc5484fbf3c4cd655a3e75d P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting consensus metadata
I20260807 18:06:42.033785 16276 catalog_manager.cc:4841] TS 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881): tablet 2568c7c47fc5484fbf3c4cd655a3e75d (table test-table [id=c6cd609ed31346858da5607caee8bce2]) successfully deleted
I20260807 18:06:42.036978 16884 ts_tablet_manager.cc:1918] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:42.037223 16884 log.cc:1196] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/2568c7c47fc5484fbf3c4cd655a3e75d
I20260807 18:06:42.038043 16884 ts_tablet_manager.cc:1939] T 2568c7c47fc5484fbf3c4cd655a3e75d P 5c4326d6cf6546208d17176ab5c0c939: Deleting consensus metadata
I20260807 18:06:42.040426 16278 catalog_manager.cc:4841] TS 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397): tablet 2568c7c47fc5484fbf3c4cd655a3e75d (table test-table [id=c6cd609ed31346858da5607caee8bce2]) successfully deleted
Aug 07 18:06:42 dist-test-slave-rnrw krb5kdc[16247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 18:06:42 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:42 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:06:42.118868 16892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:42.099402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.8.62.190:35425)
0807 18:06:42.100722 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.100729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.100785 (+ 56us) server_negotiation.cc:407] Connection header received
0807 18:06:42.100890 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.100897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.101015 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.101247 (+ 232us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:42.103698 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.105100 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.107173 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.107968 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.108288 (+ 320us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.118210 (+ 9922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33942: BlockingRecv error: recv got EOF from 127.0.0.1:33942 (error 108)
Metrics: {"server-negotiator.queue_time_us":1072,"thread_start_us":844,"threads_started":1}
Aug 07 18:06:42 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM
I20260807 18:06:42.155542 16892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.123867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33954 (local address 127.8.62.190:35425)
0807 18:06:42.124274 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.124280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.125359 (+ 1079us) server_negotiation.cc:407] Connection header received
0807 18:06:42.125736 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.125743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.125848 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.126030 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:42.129075 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.130483 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.133866 (+ 3383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.134607 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.134804 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.143836 (+ 9032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:42.143844 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:42.143848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:42.143915 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:42.149464 (+ 5549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:42.151367 (+ 1903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:42.151383 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:42.151388 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:42.151749 (+ 361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:42.152632 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:42.152640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:42.152643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:42.153370 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:42.153820 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.154401 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.154738 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260807 18:06:42.163475 16290 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:06:42.164911 16290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:42.179397 16436 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7f683497384c73a50aa6cf33179df6 (DEFAULT_TABLE table=test-table [id=d3d172a4544f4b3ea980add714a40e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.180436 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7f683497384c73a50aa6cf33179df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.181296 16728 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7f683497384c73a50aa6cf33179df6 (DEFAULT_TABLE table=test-table [id=d3d172a4544f4b3ea980add714a40e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.182113 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7f683497384c73a50aa6cf33179df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.182406 16583 tablet_service.cc:1467] Processing CreateTablet for tablet 5e7f683497384c73a50aa6cf33179df6 (DEFAULT_TABLE table=test-table [id=d3d172a4544f4b3ea980add714a40e36]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.183152 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e7f683497384c73a50aa6cf33179df6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.201732 16898 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Bootstrap starting.
I20260807 18:06:42.206573 16898 tablet_bootstrap.cc:654] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.225320 16901 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Bootstrap starting.
I20260807 18:06:42.225651 16898 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: No bootstrap required, opened a new log
I20260807 18:06:42.225826 16898 ts_tablet_manager.cc:1397] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent bootstrapping tablet: real 0.024s user 0.012s sys 0.008s
I20260807 18:06:42.226986 16898 raft_consensus.cc:348] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.227501 16898 raft_consensus.cc:374] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.227733 16898 raft_consensus.cc:729] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.228335 16898 consensus_queue.cc:260] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.229856 16900 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Bootstrap starting.
I20260807 18:06:42.230736 16898 ts_tablet_manager.cc:1428] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 18:06:42.234551 16901 tablet_bootstrap.cc:654] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.236449 16900 tablet_bootstrap.cc:654] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.242698 16901 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: No bootstrap required, opened a new log
I20260807 18:06:42.243036 16901 ts_tablet_manager.cc:1397] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.006s
I20260807 18:06:42.243257 16900 tablet_bootstrap.cc:492] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: No bootstrap required, opened a new log
I20260807 18:06:42.243623 16900 ts_tablet_manager.cc:1397] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260807 18:06:42.244437 16901 raft_consensus.cc:348] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.244940 16901 raft_consensus.cc:374] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.245193 16901 raft_consensus.cc:729] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4326d6cf6546208d17176ab5c0c939, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.245393 16900 raft_consensus.cc:348] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.245906 16900 raft_consensus.cc:374] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.246093 16900 raft_consensus.cc:729] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.245788 16901 consensus_queue.cc:260] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.247023 16901 ts_tablet_manager.cc:1428] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 18:06:42.246793 16900 consensus_queue.cc:260] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.249006 16900 ts_tablet_manager.cc:1428] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 18:06:42.373600 16831 raft_consensus.cc:482] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:42.373986 16831 raft_consensus.cc:504] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.384328 16831 leader_election.cc:288] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:42.385207 16832 raft_consensus.cc:482] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:42.385809 16832 raft_consensus.cc:504] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.389920 16832 leader_election.cc:288] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689), 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881)
I20260807 18:06:42.416771 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.377424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40397 (local address 127.8.62.131:50603)
0807 18:06:42.402351 (+ 24927us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:42.402387 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:42.402652 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:42.404677 (+ 2025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:42.404693 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:42.404754 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:42.405544 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.405569 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.408444 (+ 2875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.408462 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.411819 (+ 3357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.411834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.415239 (+ 3405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.415247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.415576 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.415630 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 18:06:42.415751 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 18:06:42.415932 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24727,"mutex_wait_us":141,"thread_start_us":25283,"threads_started":1}
I20260807 18:06:42.417503 16904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.377348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.8.62.131:58875)
0807 18:06:42.396895 (+ 19547us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:42.396949 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:42.397274 (+ 325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:42.403236 (+ 5962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:42.403245 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:42.403305 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:42.403897 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.403924 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.407522 (+ 3598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.407528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.411610 (+ 4082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.411643 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.414868 (+ 3225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.414886 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.415248 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.415325 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 18:06:42.415477 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:06:42.415757 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18112,"mutex_wait_us":123,"thread_start_us":18499,"threads_started":1}
I20260807 18:06:42.419158 16907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.378246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:58875 (local address 127.8.62.129:42689)
0807 18:06:42.402187 (+ 23941us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.402193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.402229 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:06:42.402367 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.402386 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.402564 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.404238 (+ 1674us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:42.404673 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.407039 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.411907 (+ 4868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.414459 (+ 2552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.416994 (+ 2535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.417184 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.417287 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.418316 (+ 1029us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23716,"thread_start_us":28945,"threads_started":1}
I20260807 18:06:42.420159 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.378249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:50603 (local address 127.8.62.130:40397)
0807 18:06:42.403551 (+ 25302us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.403570 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.403610 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:06:42.403761 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.403771 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.403947 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.404141 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:42.406024 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.408131 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.412079 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.414412 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.414842 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.415005 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.417679 (+ 2674us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.418013 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25052,"thread_start_us":24822,"threads_started":1}
I20260807 18:06:42.420975 16456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" is_pre_election: true
I20260807 18:06:42.421948 16456 raft_consensus.cc:2454] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 0.
I20260807 18:06:42.423388 16677 leader_election.cc:302] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:42.424641 16831 raft_consensus.cc:2781] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:42.424940 16831 raft_consensus.cc:482] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:42.425230 16831 raft_consensus.cc:3037] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.433270 16831 raft_consensus.cc:504] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.437359 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.396473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:60973 (local address 127.8.62.131:32881)
0807 18:06:42.401385 (+ 4912us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.401404 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.421300 (+ 19896us) server_negotiation.cc:407] Connection header received
0807 18:06:42.421780 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.421788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.421934 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.422076 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:42.423765 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.426183 (+ 2418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.431901 (+ 5718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.434547 (+ 2646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.435117 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.435368 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.436310 (+ 942us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.436692 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4552,"thread_start_us":4862,"threads_started":1}
I20260807 18:06:42.437150 16456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1807e3d1a654e4d9a03607bf5b71d10"
I20260807 18:06:42.437674 16456 raft_consensus.cc:3037] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.438818 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.390745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:32881 (local address 127.8.62.130:60973)
0807 18:06:42.421097 (+ 30352us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:42.421133 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:42.421481 (+ 348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:42.422674 (+ 1193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:42.422693 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:42.422748 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:42.423386 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.423401 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.426689 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.426698 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.431326 (+ 4628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.431386 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.435322 (+ 3936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.435333 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.435846 (+ 513us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.435943 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 18:06:42.437802 (+ 1859us) client_negotiation.cc:240] Negotiation successful
0807 18:06:42.438138 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30114,"thread_start_us":29852,"threads_started":1}
I20260807 18:06:42.442023 16748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "5c4326d6cf6546208d17176ab5c0c939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" is_pre_election: true
I20260807 18:06:42.442900 16748 raft_consensus.cc:2379] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c4326d6cf6546208d17176ab5c0c939 in current term 1: Already voted for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in this term.
I20260807 18:06:42.452102 16456 raft_consensus.cc:2454] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 1.
I20260807 18:06:42.453393 16677 leader_election.cc:302] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:42.456347 16912 raft_consensus.cc:2781] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:42.456679 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.404059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:36197 (local address 127.8.62.129:42689)
0807 18:06:42.420400 (+ 16341us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.420422 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.440161 (+ 19739us) server_negotiation.cc:407] Connection header received
0807 18:06:42.440796 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.440803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.440991 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.441122 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:06:42.442272 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.444342 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.449469 (+ 5127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.452292 (+ 2823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.452720 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.452895 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.454264 (+ 1369us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.454621 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15992,"thread_start_us":15733,"threads_started":1}
I20260807 18:06:42.457093 16831 leader_election.cc:288] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 election: Requested vote from peers b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:42.458841 16912 raft_consensus.cc:686] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 LEADER]: Becoming Leader. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Running, Role: LEADER
I20260807 18:06:42.458997 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.416912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.8.62.130:36197)
0807 18:06:42.439843 (+ 22931us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:42.439902 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:42.440482 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:42.441519 (+ 1037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:42.441525 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:42.441581 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:06:42.441936 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.441952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.444802 (+ 2850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.444810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.448915 (+ 4105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:42.448972 (+ 57us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.453266 (+ 4294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.453276 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:42.453740 (+ 464us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.453837 (+ 97us) client_negotiation.cc:769] Sending connection context
0807 18:06:42.457318 (+ 3481us) client_negotiation.cc:240] Negotiation successful
0807 18:06:42.457827 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20995,"thread_start_us":39636,"threads_started":1}
I20260807 18:06:42.459733 16912 consensus_queue.cc:237] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.460304 16608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939" is_pre_election: true
I20260807 18:06:42.460840 16608 raft_consensus.cc:2454] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 0.
I20260807 18:06:42.460471 16607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939"
I20260807 18:06:42.461278 16607 raft_consensus.cc:3037] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.463464 16456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e7f683497384c73a50aa6cf33179df6" candidate_uuid: "5c4326d6cf6546208d17176ab5c0c939" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" is_pre_election: true
I20260807 18:06:42.464330 16456 raft_consensus.cc:2379] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5c4326d6cf6546208d17176ab5c0c939 in current term 1: Already voted for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in this term.
I20260807 18:06:42.465974 16532 leader_election.cc:302] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c4326d6cf6546208d17176ab5c0c939; no voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10
I20260807 18:06:42.468519 16607 raft_consensus.cc:2454] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 1.
I20260807 18:06:42.468969 16832 raft_consensus.cc:2726] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 18:06:42.471665 16290 catalog_manager.cc:5495] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } health_report { overall_health: HEALTHY } } }
I20260807 18:06:42.551098 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.540481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.8.62.131:32881)
0807 18:06:42.540912 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:42.540934 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:42.540983 (+ 49us) server_negotiation.cc:407] Connection header received
0807 18:06:42.541483 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:42.541490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:42.541663 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:42.541996 (+ 333us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:42.542874 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.544140 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.546801 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:42.547662 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:42.547960 (+ 298us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:42.548920 (+ 960us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:42.549893 (+ 973us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:42.550081 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:42.550355 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:06:42.550533 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190}
I20260807 18:06:42.559156 16607 raft_consensus.cc:1261] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Refusing update from remote peer 146d5c2a01e54432a6f1bd4a6de18fc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:42.559264 16456 raft_consensus.cc:1261] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Refusing update from remote peer 146d5c2a01e54432a6f1bd4a6de18fc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:42.560326 16831 consensus_queue.cc:1035] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:42.561025 16912 consensus_queue.cc:1035] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:42.637667 16292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:42.638265 16292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:42.648334 16292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Sending DeleteTablet for 3 replicas of tablet 5e7f683497384c73a50aa6cf33179df6
I20260807 18:06:42.650688 16436 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7f683497384c73a50aa6cf33179df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:42 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40658
I20260807 18:06:42.651726 16583 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7f683497384c73a50aa6cf33179df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:42 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58704
I20260807 18:06:42.652155 16926 tablet_replica.cc:330] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: stopping tablet replica
I20260807 18:06:42.652585 16926 raft_consensus.cc:2229] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:42.653155 16926 raft_consensus.cc:2258] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.653709 16728 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e7f683497384c73a50aa6cf33179df6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:42 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52274
I20260807 18:06:42.655046 16292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 18:06:42.655689 16926 ts_tablet_manager.cc:1905] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 18:06:42.656491 16292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:06:42.669157 16927 tablet_replica.cc:330] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: stopping tablet replica
I20260807 18:06:42.670482 16928 tablet_replica.cc:330] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: stopping tablet replica
I20260807 18:06:42.671113 16927 raft_consensus.cc:2229] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:42.676910 16436 tablet_service.cc:1467] Processing CreateTablet for tablet e495d64c83e548e2b5002949db8e1ca4 (DEFAULT_TABLE table=test-table [id=0fa0f4603c0b4aecbe22383bb696126d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.677719 16436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e495d64c83e548e2b5002949db8e1ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.678174 16928 raft_consensus.cc:2229] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:42.677717 16927 raft_consensus.cc:2258] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.679742 16928 raft_consensus.cc:2258] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:42.679786 16926 ts_tablet_manager.cc:1918] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:42.680050 16926 log.cc:1196] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/5e7f683497384c73a50aa6cf33179df6
I20260807 18:06:42.681138 16926 ts_tablet_manager.cc:1939] T 5e7f683497384c73a50aa6cf33179df6 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting consensus metadata
I20260807 18:06:42.682448 16927 ts_tablet_manager.cc:1905] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:42.683666 16583 tablet_service.cc:1467] Processing CreateTablet for tablet e495d64c83e548e2b5002949db8e1ca4 (DEFAULT_TABLE table=test-table [id=0fa0f4603c0b4aecbe22383bb696126d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.684576 16583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e495d64c83e548e2b5002949db8e1ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.689242 16277 catalog_manager.cc:4841] TS b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689): tablet 5e7f683497384c73a50aa6cf33179df6 (table test-table [id=d3d172a4544f4b3ea980add714a40e36]) successfully deleted
I20260807 18:06:42.691980 16728 tablet_service.cc:1467] Processing CreateTablet for tablet e495d64c83e548e2b5002949db8e1ca4 (DEFAULT_TABLE table=test-table [id=0fa0f4603c0b4aecbe22383bb696126d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:06:42.692890 16728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e495d64c83e548e2b5002949db8e1ca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:42.697309 16928 ts_tablet_manager.cc:1905] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:42.708870 16927 ts_tablet_manager.cc:1918] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:42.709360 16927 log.cc:1196] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/5e7f683497384c73a50aa6cf33179df6
I20260807 18:06:42.710430 16927 ts_tablet_manager.cc:1939] T 5e7f683497384c73a50aa6cf33179df6 P 5c4326d6cf6546208d17176ab5c0c939: Deleting consensus metadata
I20260807 18:06:42.714262 16901 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Bootstrap starting.
I20260807 18:06:42.715691 16898 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Bootstrap starting.
I20260807 18:06:42.718367 16901 tablet_bootstrap.cc:654] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.720064 16898 tablet_bootstrap.cc:654] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.729944 16278 catalog_manager.cc:4841] TS 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397): tablet 5e7f683497384c73a50aa6cf33179df6 (table test-table [id=d3d172a4544f4b3ea980add714a40e36]) successfully deleted
I20260807 18:06:42.740996 16900 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Bootstrap starting.
I20260807 18:06:42.744181 16900 tablet_bootstrap.cc:654] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:42.755038 16900 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: No bootstrap required, opened a new log
I20260807 18:06:42.755460 16900 ts_tablet_manager.cc:1397] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.000s
I20260807 18:06:42.757596 16900 raft_consensus.cc:348] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.758363 16900 raft_consensus.cc:374] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.758636 16900 raft_consensus.cc:729] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1807e3d1a654e4d9a03607bf5b71d10, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.759465 16900 consensus_queue.cc:260] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.768633 16898 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: No bootstrap required, opened a new log
I20260807 18:06:42.769104 16898 ts_tablet_manager.cc:1397] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent bootstrapping tablet: real 0.054s user 0.011s sys 0.000s
I20260807 18:06:42.770725 16898 raft_consensus.cc:348] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.771204 16900 ts_tablet_manager.cc:1428] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260807 18:06:42.771425 16898 raft_consensus.cc:374] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.771689 16898 raft_consensus.cc:729] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.773105 16898 consensus_queue.cc:260] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.784087 16898 ts_tablet_manager.cc:1428] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Time spent starting tablet: real 0.015s user 0.003s sys 0.000s
I20260807 18:06:42.802441 16928 ts_tablet_manager.cc:1918] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:06:42.802888 16928 log.cc:1196] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/5e7f683497384c73a50aa6cf33179df6
I20260807 18:06:42.804143 16928 ts_tablet_manager.cc:1939] T 5e7f683497384c73a50aa6cf33179df6 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting consensus metadata
I20260807 18:06:42.808466 16276 catalog_manager.cc:4841] TS 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881): tablet 5e7f683497384c73a50aa6cf33179df6 (table test-table [id=d3d172a4544f4b3ea980add714a40e36]) successfully deleted
I20260807 18:06:42.810276 16901 tablet_bootstrap.cc:492] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: No bootstrap required, opened a new log
I20260807 18:06:42.810616 16901 ts_tablet_manager.cc:1397] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Time spent bootstrapping tablet: real 0.097s user 0.036s sys 0.051s
I20260807 18:06:42.812222 16901 raft_consensus.cc:348] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.812562 16901 raft_consensus.cc:374] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:42.812700 16901 raft_consensus.cc:729] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c4326d6cf6546208d17176ab5c0c939, State: Initialized, Role: FOLLOWER
I20260807 18:06:42.813174 16901 consensus_queue.cc:260] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.814406 16901 ts_tablet_manager.cc:1428] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 18:06:42.871397 16831 raft_consensus.cc:482] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:06:42.871867 16831 raft_consensus.cc:504] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.873459 16831 leader_election.cc:288] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:42.874930 16456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e495d64c83e548e2b5002949db8e1ca4" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" is_pre_election: true
I20260807 18:06:42.875422 16456 raft_consensus.cc:2454] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 0.
I20260807 18:06:42.875458 16607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e495d64c83e548e2b5002949db8e1ca4" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939" is_pre_election: true
I20260807 18:06:42.876060 16607 raft_consensus.cc:2454] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 0.
I20260807 18:06:42.876534 16677 leader_election.cc:302] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:42.877359 16831 raft_consensus.cc:2781] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:06:42.877542 16831 raft_consensus.cc:482] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:06:42.877763 16831 raft_consensus.cc:3037] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.883404 16831 raft_consensus.cc:504] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.885759 16456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e495d64c83e548e2b5002949db8e1ca4" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1807e3d1a654e4d9a03607bf5b71d10"
I20260807 18:06:42.886162 16456 raft_consensus.cc:3037] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.887723 16607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e495d64c83e548e2b5002949db8e1ca4" candidate_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c4326d6cf6546208d17176ab5c0c939"
I20260807 18:06:42.888185 16607 raft_consensus.cc:3037] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:42.893219 16456 raft_consensus.cc:2454] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 1.
I20260807 18:06:42.894348 16677 leader_election.cc:302] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 146d5c2a01e54432a6f1bd4a6de18fc6, b1807e3d1a654e4d9a03607bf5b71d10; no voters:
I20260807 18:06:42.895162 16912 raft_consensus.cc:2781] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:42.895220 16607 raft_consensus.cc:2454] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 146d5c2a01e54432a6f1bd4a6de18fc6 in term 1.
I20260807 18:06:42.895488 16912 raft_consensus.cc:686] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 LEADER]: Becoming Leader. State: Replica: 146d5c2a01e54432a6f1bd4a6de18fc6, State: Running, Role: LEADER
I20260807 18:06:42.896682 16831 leader_election.cc:288] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [CANDIDATE]: Term 1 election: Requested vote from peers b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689), 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397)
I20260807 18:06:42.898089 16912 consensus_queue.cc:237] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } }
I20260807 18:06:42.903651 16292 catalog_manager.cc:5495] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "146d5c2a01e54432a6f1bd4a6de18fc6" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 32881 } health_report { overall_health: HEALTHY } } }
I20260807 18:06:43.017911 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.993137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:34635 (local address 127.8.62.129:42689)
0807 18:06:43.005157 (+ 12020us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:43.005167 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:43.005210 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:06:43.005361 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:43.005381 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:43.005590 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:43.005797 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:43.006860 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.009242 (+ 2382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.012764 (+ 3522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.014231 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.014572 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.015852 (+ 1280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:43.016190 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:43.016403 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:43.016811 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 18:06:43.017010 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11736,"thread_start_us":11975,"threads_started":1}
I20260807 18:06:43.018034 16938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:42.993270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:42689 (local address 127.8.62.131:34635)
0807 18:06:43.004521 (+ 11251us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:43.004596 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:43.004922 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:43.005842 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:43.005850 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:43.005870 (+ 20us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:06:43.006536 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:43.006566 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.009499 (+ 2933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.009505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:43.012425 (+ 2920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:43.012462 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.014762 (+ 2300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.014773 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:43.015169 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.015392 (+ 223us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:06:43.016409 (+ 1017us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:06:43.016465 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 18:06:43.016722 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 18:06:43.017093 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10844,"thread_start_us":10478,"threads_started":1}
I20260807 18:06:43.042691 16456 raft_consensus.cc:1261] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Refusing update from remote peer 146d5c2a01e54432a6f1bd4a6de18fc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:43.042757 16607 raft_consensus.cc:1261] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Refusing update from remote peer 146d5c2a01e54432a6f1bd4a6de18fc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:06:43.044011 16912 consensus_queue.cc:1035] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c4326d6cf6546208d17176ab5c0c939" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:43.044691 16831 consensus_queue.cc:1035] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1807e3d1a654e4d9a03607bf5b71d10" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:06:43.151435 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:43.123536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:32881 (local address 127.8.62.129:51953)
0807 18:06:43.134273 (+ 10737us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:43.134310 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:43.134654 (+ 344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:43.136700 (+ 2046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:43.136718 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:43.136750 (+ 32us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 18:06:43.137474 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:43.137500 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.141003 (+ 3503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.141037 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:43.145304 (+ 4267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:43.145344 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.148511 (+ 3167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.148521 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:43.148975 (+ 454us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.149107 (+ 132us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 18:06:43.150166 (+ 1059us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 18:06:43.150191 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:06:43.150338 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:06:43.150717 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9713,"thread_start_us":9431,"threads_started":1}
I20260807 18:06:43.152810 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:43.124391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:51953 (local address 127.8.62.131:32881)
0807 18:06:43.131418 (+ 7027us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:43.131426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:43.134882 (+ 3456us) server_negotiation.cc:407] Connection header received
0807 18:06:43.135079 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:43.135096 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:43.135285 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:43.135460 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:06:43.137876 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.140669 (+ 2793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.145675 (+ 5006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.147716 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.148207 (+ 491us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.149533 (+ 1326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:06:43.149843 (+ 310us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:06:43.150063 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:43.151984 (+ 1921us) server_negotiation.cc:299] Negotiation successful
0807 18:06:43.152147 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6764,"thread_start_us":7116,"threads_started":1}
I20260807 18:06:43.291116 16291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:43.291745 16291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33954:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:06:43.298976 16291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ef6379433061441ab2fdc0151302d424: Sending DeleteTablet for 3 replicas of tablet e495d64c83e548e2b5002949db8e1ca4
I20260807 18:06:43.300724 16436 tablet_service.cc:1514] Processing DeleteTablet for tablet e495d64c83e548e2b5002949db8e1ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:43 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40658
I20260807 18:06:43.300800 16583 tablet_service.cc:1514] Processing DeleteTablet for tablet e495d64c83e548e2b5002949db8e1ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:43 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:58704
I20260807 18:06:43.303972 16728 tablet_service.cc:1514] Processing DeleteTablet for tablet e495d64c83e548e2b5002949db8e1ca4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:06:43 UTC) from {username='oryx', principal='oryx/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:52274
I20260807 18:06:43.304646 16928 tablet_replica.cc:330] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: stopping tablet replica
I20260807 18:06:43.305070 16928 raft_consensus.cc:2229] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:06:43.305791 16928 raft_consensus.cc:2258] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:43.307529 16948 tablet_replica.cc:330] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: stopping tablet replica
I20260807 18:06:43.307984 16928 ts_tablet_manager.cc:1905] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:43.308027 16948 raft_consensus.cc:2229] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:43.308684 16948 raft_consensus.cc:2258] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:43.311558 16948 ts_tablet_manager.cc:1905] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:43.316169 16947 tablet_replica.cc:330] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: stopping tablet replica
I20260807 18:06:43.316646 16947 raft_consensus.cc:2229] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:06:43.317119 16947 raft_consensus.cc:2258] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:06:43.318785 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 16368
I20260807 18:06:43.319511 16947 ts_tablet_manager.cc:1905] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:06:43.335510 16947 ts_tablet_manager.cc:1918] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:43.335821 16947 log.cc:1196] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/e495d64c83e548e2b5002949db8e1ca4
I20260807 18:06:43.336862 16947 ts_tablet_manager.cc:1939] T e495d64c83e548e2b5002949db8e1ca4 P b1807e3d1a654e4d9a03607bf5b71d10: Deleting consensus metadata
I20260807 18:06:43.337642 16928 ts_tablet_manager.cc:1918] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:43.337950 16928 log.cc:1196] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/e495d64c83e548e2b5002949db8e1ca4
I20260807 18:06:43.338907 16928 ts_tablet_manager.cc:1939] T e495d64c83e548e2b5002949db8e1ca4 P 146d5c2a01e54432a6f1bd4a6de18fc6: Deleting consensus metadata
I20260807 18:06:43.339128 16277 catalog_manager.cc:4841] TS b1807e3d1a654e4d9a03607bf5b71d10 (127.8.62.129:42689): tablet e495d64c83e548e2b5002949db8e1ca4 (table test-table [id=0fa0f4603c0b4aecbe22383bb696126d]) successfully deleted
I20260807 18:06:43.343216 16948 ts_tablet_manager.cc:1918] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 18:06:43.343498 16948 log.cc:1196] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/e495d64c83e548e2b5002949db8e1ca4
I20260807 18:06:43.344367 16948 ts_tablet_manager.cc:1939] T e495d64c83e548e2b5002949db8e1ca4 P 5c4326d6cf6546208d17176ab5c0c939: Deleting consensus metadata
I20260807 18:06:43.346680 16278 catalog_manager.cc:4841] TS 5c4326d6cf6546208d17176ab5c0c939 (127.8.62.130:40397): tablet e495d64c83e548e2b5002949db8e1ca4 (table test-table [id=0fa0f4603c0b4aecbe22383bb696126d]) successfully deleted
I20260807 18:06:43.347724 16276 catalog_manager.cc:4841] TS 146d5c2a01e54432a6f1bd4a6de18fc6 (127.8.62.131:32881): tablet e495d64c83e548e2b5002949db8e1ca4 (table test-table [id=0fa0f4603c0b4aecbe22383bb696126d]) successfully deleted
Aug 07 18:06:43 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.129@KRBTEST.COM
I20260807 18:06:43.371351 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:43.322669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.8.62.129:42689)
0807 18:06:43.323058 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:43.323075 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:43.341990 (+ 18915us) server_negotiation.cc:407] Connection header received
0807 18:06:43.342802 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:43.342808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:43.342987 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:43.343153 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:43.346730 (+ 3577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.348582 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.350623 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.351415 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.351730 (+ 315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.359456 (+ 7726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:43.359495 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:43.359526 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:43.359608 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:43.365030 (+ 5422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.366508 (+ 1478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.366533 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.366537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.366859 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.367918 (+ 1059us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.367923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.367927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.368810 (+ 883us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:43.369339 (+ 529us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:43.369924 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 18:06:43.370489 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 18:06:43.372749 16504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:43.606664 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 16368
I20260807 18:06:43.653493 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 16514
Aug 07 18:06:43 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.130@KRBTEST.COM
I20260807 18:06:43.683871 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:43.654714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.8.62.130:40397)
0807 18:06:43.655486 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:43.655491 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:43.655516 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:06:43.655607 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:43.655612 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:43.655690 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:43.655959 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:43.658723 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.659880 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.662112 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.663084 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.663343 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.672426 (+ 9083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:43.672454 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:43.672484 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:43.672840 (+ 356us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:43.678431 (+ 5591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.679838 (+ 1407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.679856 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.679859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.680110 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.680874 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.680879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.680881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.681541 (+ 660us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:43.682188 (+ 647us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:43.682750 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 18:06:43.683236 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":383,"threads_started":1}
I20260807 18:06:43.685328 16650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:43.846877 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 16514
I20260807 18:06:43.881724 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 16660
Aug 07 18:06:43 dist-test-slave-rnrw krb5kdc[16247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.62.131@KRBTEST.COM
I20260807 18:06:43.913870 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:43.882824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52296 (local address 127.8.62.131:32881)
0807 18:06:43.883625 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:43.883632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:43.883665 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:06:43.883770 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:43.883775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:43.883896 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:43.884055 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:43.886441 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.887637 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.890161 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:43.890988 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:43.891227 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:43.902347 (+ 11120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:43.902382 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:43.902410 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:43.902520 (+ 110us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:43.907897 (+ 5377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.909706 (+ 1809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.909718 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.909721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.909982 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:43.910704 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:43.910710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:43.910721 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:43.911689 (+ 968us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:43.912202 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:43.912909 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 18:06:43.913321 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":407,"threads_started":1}
I20260807 18:06:43.915140 16796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:44.094276 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 16660
I20260807 18:06:44.131211 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 16260
I20260807 18:06:44.133311 16336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:44.302101 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 16260
2026-08-07T18:06:44Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7275 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:44 dist-test-slave-rnrw krb5kdc[16958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:44 dist-test-slave-rnrw krb5kdc[16958](info): set up 2 sockets
Aug 07 18:06:44 dist-test-slave-rnrw krb5kdc[16958](info): commencing operation
krb5kdc: starting...
W20260807 18:06:45.584759 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:45 dist-test-slave-rnrw krb5kdc[16958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:45.830439 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33657
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:33657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:46.178282 16971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:46.178634 16971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:46.178731 16971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:46.186208 16971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:46.186339 16971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:46.186388 16971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:46.186451 16971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:46.186493 16971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:46.196722 16971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:33657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:46.198074 16971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:46.199349 16971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:46.212400 16976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:46.212502 16977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:46.212363 16979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:46.212968 16971 server_base.cc:1034] running on GCE node
I20260807 18:06:46.213869 16971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:46.215405 16971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:46.216686 16971 hybrid_clock.cc:648] HybridClock initialized: now 1786126006216615 us; error 54 us; skew 500 ppm
I20260807 18:06:46.217101 16971 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T18:06:46Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1966 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:06:46 dist-test-slave-rnrw krb5kdc[16985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:06:46 dist-test-slave-rnrw krb5kdc[16985](info): set up 2 sockets
Aug 07 18:06:46 dist-test-slave-rnrw krb5kdc[16985](info): commencing operation
krb5kdc: starting...
W20260807 18:06:47.412652 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:06:47 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:06:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:06:47.660259 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33997
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:33997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:47.996280 16998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:47.996634 16998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:47.996801 16998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:48.004854 16998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:48.004980 16998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:48.005059 16998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:48.005151 16998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:48.005223 16998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:48.015523 16998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:33997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:48.016997 16998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:48.018369 16998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:48.032871 17003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:48.032864 17007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:48.032912 17004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:48.033783 16998 server_base.cc:1034] running on GCE node
I20260807 18:06:48.034768 16998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:48.036449 16998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:48.037717 16998 hybrid_clock.cc:648] HybridClock initialized: now 1786126008037670 us; error 36 us; skew 500 ppm
I20260807 18:06:48.038133 16998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:48 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126008, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:48.045779 16998 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:48.047518 16998 webserver.cc:460] Webserver started at http://127.8.62.190:42707/ using document root <none> and password file <none>
I20260807 18:06:48.048274 16998 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:48.048419 16998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:48.048803 16998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:48.051918 16998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "f1e6bf6e19f545849fa1486ff8cb8763"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "02b0154fd1c5d956abe169b7807d563f"
server_key_iv: "db40cd1b1df12a52196673f6cef8bbfc"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.052862 16998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "f1e6bf6e19f545849fa1486ff8cb8763"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "02b0154fd1c5d956abe169b7807d563f"
server_key_iv: "db40cd1b1df12a52196673f6cef8bbfc"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.058599 16998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:06:48.062894 17016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:48.064625 16998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:06:48.064833 16998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "f1e6bf6e19f545849fa1486ff8cb8763"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "02b0154fd1c5d956abe169b7807d563f"
server_key_iv: "db40cd1b1df12a52196673f6cef8bbfc"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.065065 16998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:48.099273 16998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:48.104152 16998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:48.104465 16998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:48.124517 16998 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:33997
I20260807 18:06:48.124595 17074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:33997 every 8 connection(s)
I20260807 18:06:48.126051 16998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:48.130175 17076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:06:48.134383 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 16998
I20260807 18:06:48.134608 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:06:48.135274 8442 external_mini_cluster.cc:1428] Setting key 289a3f65fbeff37c81cb439daa577c15
I20260807 18:06:48.142028 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Bootstrap starting.
I20260807 18:06:48.146078 17076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Neither blocks nor log segments found. Creating new log.
I20260807 18:06:48.147372 17076 log.cc:824] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:48.151400 17076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: No bootstrap required, opened a new log
Aug 07 18:06:48 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:48.160094 17076 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:48.160581 17076 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:06:48.160724 17076 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e6bf6e19f545849fa1486ff8cb8763, State: Initialized, Role: FOLLOWER
I20260807 18:06:48.161387 17076 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:48.161610 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:48.137711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.8.62.190:33997)
0807 18:06:48.138601 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:48.138613 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:48.138666 (+ 53us) server_negotiation.cc:407] Connection header received
0807 18:06:48.140663 (+ 1997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:48.140689 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:48.141131 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:48.141487 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:48.143500 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.145020 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.146910 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.147782 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.148000 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:48.153163 (+ 5163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:48.153175 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:48.153203 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:48.153246 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:48.156745 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:48.157525 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:48.157539 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:48.157550 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:48.157742 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:48.158153 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:48.158162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:48.158170 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:48.158858 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:48.159224 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:48.159692 (+ 468us) server_negotiation.cc:299] Negotiation successful
0807 18:06:48.160156 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":397,"threads_started":1}
I20260807 18:06:48.161660 17076 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:48.161790 17076 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:48.162009 17076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:06:48.164439 17076 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:48.165025 17076 leader_election.cc:302] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e6bf6e19f545849fa1486ff8cb8763; no voters:
I20260807 18:06:48.165596 17076 leader_election.cc:288] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:06:48.165776 17081 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:06:48.166461 17081 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 LEADER]: Becoming Leader. State: Replica: f1e6bf6e19f545849fa1486ff8cb8763, State: Running, Role: LEADER
I20260807 18:06:48.167017 17081 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:48.168143 17076 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:48.170234 17083 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1e6bf6e19f545849fa1486ff8cb8763. Latest consensus state: current_term: 1 leader_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } } }
I20260807 18:06:48.170742 17083 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:48.170261 17082 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } } }
I20260807 18:06:48.171236 17082 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:48.177201 17088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:48.183065 17088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:48.193331 17088 catalog_manager.cc:1347] Generated new cluster ID: a54d762411694799af04aba63d0bebc2
I20260807 18:06:48.193464 17088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:48.214589 17088 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:06:48.216022 17088 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:48.225729 17088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Generated new TSK 0
I20260807 18:06:48.226928 17088 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:06:48.296844 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:48.654034 17103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:48.654372 17103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:48.654577 17103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:48.661914 17103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:48.662068 17103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:48.662246 17103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:48.672670 17103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:48.674046 17103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:48.675369 17103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:48.689216 17111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:48.689297 17109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:48.689266 17108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:48.690244 17103 server_base.cc:1034] running on GCE node
I20260807 18:06:48.690866 17103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:48.691864 17103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:48.693095 17103 hybrid_clock.cc:648] HybridClock initialized: now 1786126008693022 us; error 65 us; skew 500 ppm
I20260807 18:06:48.693416 17103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:48 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126008, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:48.696798 17103 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:06:48.698377 17103 webserver.cc:460] Webserver started at http://127.8.62.129:37335/ using document root <none> and password file <none>
I20260807 18:06:48.699098 17103 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:48.699244 17103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:48.699565 17103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:48.702977 17103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "989db0744ede418f98f7e3944f989eec"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "63e97cd7f6c54dfc4f420a27be2a960a"
server_key_iv: "e5c39dda10ac977bc52c9e79a3f201ed"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.703850 17103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "989db0744ede418f98f7e3944f989eec"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "63e97cd7f6c54dfc4f420a27be2a960a"
server_key_iv: "e5c39dda10ac977bc52c9e79a3f201ed"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.709710 17103 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:06:48.713449 17118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:48.714877 17103 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:06:48.715056 17103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "989db0744ede418f98f7e3944f989eec"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "63e97cd7f6c54dfc4f420a27be2a960a"
server_key_iv: "e5c39dda10ac977bc52c9e79a3f201ed"
server_key_version: "encryptionkey@0"
I20260807 18:06:48.715256 17103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:48.728421 17103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:48.732442 17103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:48.732764 17103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:48.733667 17103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:48.735050 17103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:48.735126 17103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:48.735210 17103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:48.735299 17103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:48.769945 17103 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:46845
I20260807 18:06:48.770017 17234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:46845 every 8 connection(s)
I20260807 18:06:48.771312 17103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:48.774163 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17103
I20260807 18:06:48.774382 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:06:48.775157 8442 external_mini_cluster.cc:1428] Setting key 49c356fddcef67d66568200d9400bc20
Aug 07 18:06:48 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126008, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:48.804030 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:48.774749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:37573)
0807 18:06:48.775709 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:48.775759 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:48.779008 (+ 3249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:48.782155 (+ 3147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:48.782169 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:48.783052 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:48.783979 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:48.784011 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.785732 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.785741 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:48.788025 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:48.788045 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.789202 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.789212 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:48.789473 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:48.790196 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:48.790234 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:48.793565 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:48.798401 (+ 4836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:48.798414 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:48.798436 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:48.799200 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:48.799681 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:48.799689 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:48.799696 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:48.799983 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:48.800813 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:48.800823 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:48.801337 (+ 514us) client_negotiation.cc:769] Sending connection context
0807 18:06:48.801691 (+ 354us) client_negotiation.cc:240] Negotiation successful
0807 18:06:48.802350 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":390,"threads_started":1}
I20260807 18:06:48.805514 17237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:48.774122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:37573 (local address 127.8.62.190:33997)
0807 18:06:48.774748 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:48.774755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:48.781248 (+ 6493us) server_negotiation.cc:407] Connection header received
0807 18:06:48.781358 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:48.781365 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:48.781478 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:48.781614 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:48.784221 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.785569 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.788229 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:48.788941 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:48.789089 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:48.793868 (+ 4779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:48.793877 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:48.793885 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:48.793933 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:48.798199 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:48.799365 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:48.799374 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:48.799381 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:48.799544 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:48.800132 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:48.800140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:48.800146 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:48.800588 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:48.800728 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:48.804776 (+ 4048us) server_negotiation.cc:299] Negotiation successful
0807 18:06:48.805054 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":333,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:06:48.807227 17235 heartbeater.cc:347] Connected to a master server at 127.8.62.190:33997
I20260807 18:06:48.807655 17235 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:48.808482 17235 heartbeater.cc:511] Master 127.8.62.190:33997 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:06:48.810876 17036 ts_manager.cc:194] Registered new tserver with Master: 989db0744ede418f98f7e3944f989eec (127.8.62.129:46845)
I20260807 18:06:48.812381 17036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:37573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:06:48.840188 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:49.184433 17242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:49.184931 17242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:49.185048 17242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:49.192755 17242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:49.192885 17242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:49.193007 17242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:49.203233 17242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:49.204717 17242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:49.206007 17242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:49.220125 17250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:49.220268 17248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:49.220902 17247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:49.221067 17242 server_base.cc:1034] running on GCE node
I20260807 18:06:49.221747 17242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:49.222704 17242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:49.223937 17242 hybrid_clock.cc:648] HybridClock initialized: now 1786126009223849 us; error 62 us; skew 500 ppm
I20260807 18:06:49.224265 17242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:49 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:49.227834 17242 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:06:49.229375 17242 webserver.cc:460] Webserver started at http://127.8.62.130:35519/ using document root <none> and password file <none>
I20260807 18:06:49.230073 17242 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:49.230180 17242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:49.230438 17242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:49.233587 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "85b2cf62d87648eaaf581763502a5b45"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "1bfe7dcabcf5dca58c80bd837d3f4c7f"
server_key_iv: "a6d996ca45017aabdd642e51cf9d5b7a"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.234483 17242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "85b2cf62d87648eaaf581763502a5b45"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "1bfe7dcabcf5dca58c80bd837d3f4c7f"
server_key_iv: "a6d996ca45017aabdd642e51cf9d5b7a"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.240176 17242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:06:49.243760 17257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.245291 17242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:49.245471 17242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "85b2cf62d87648eaaf581763502a5b45"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "1bfe7dcabcf5dca58c80bd837d3f4c7f"
server_key_iv: "a6d996ca45017aabdd642e51cf9d5b7a"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.245693 17242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:49.275923 17242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:49.280393 17242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:49.280758 17242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:49.281822 17242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:49.283252 17242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:49.283327 17242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.283492 17242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:49.283579 17242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.318141 17242 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:34845
I20260807 18:06:49.318192 17373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:34845 every 8 connection(s)
I20260807 18:06:49.319623 17242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:49.326344 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17242
I20260807 18:06:49.326527 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:06:49.326993 8442 external_mini_cluster.cc:1428] Setting key 31d457e096dff68fa6aa97a957156655
Aug 07 18:06:49 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:49.341466 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:49.322455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:50519 (local address 127.8.62.190:33997)
0807 18:06:49.322904 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:49.322908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:49.323954 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 18:06:49.325924 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:49.325930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:49.326019 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:49.326157 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:49.327989 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.328888 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.330449 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.330892 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.331025 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:49.334605 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:49.334610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:49.334613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:49.334646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:49.337799 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:49.338781 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:49.338786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:49.338788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:49.338951 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:49.339557 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:49.339561 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:49.339564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:49.339854 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:49.339990 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:49.340969 (+ 979us) server_negotiation.cc:299] Negotiation successful
0807 18:06:49.341166 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":232,"threads_started":1}
I20260807 18:06:49.343021 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:49.323034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:50519)
0807 18:06:49.323784 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:49.323823 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:49.325637 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:49.326413 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:49.326421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:49.327032 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:49.327813 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:49.327836 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.329041 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.329047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:49.330298 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:49.330310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.331134 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.331142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:49.331380 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:49.332069 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:49.332092 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:49.334354 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:49.337951 (+ 3597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:49.337964 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:49.337986 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:49.338632 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:49.339079 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:49.339086 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:49.339093 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:49.339361 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:49.339993 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:49.340004 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:49.340511 (+ 507us) client_negotiation.cc:769] Sending connection context
0807 18:06:49.340877 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 18:06:49.341483 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":286,"threads_started":1}
I20260807 18:06:49.344650 17374 heartbeater.cc:347] Connected to a master server at 127.8.62.190:33997
I20260807 18:06:49.344972 17374 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:49.345701 17374 heartbeater.cc:511] Master 127.8.62.190:33997 requested a full tablet report, sending...
I20260807 18:06:49.347137 17036 ts_manager.cc:194] Registered new tserver with Master: 85b2cf62d87648eaaf581763502a5b45 (127.8.62.130:34845)
I20260807 18:06:49.348021 17036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:50519
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:06:49.383379 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:49.720224 17381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:49.720669 17381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:49.720834 17381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:49.728309 17381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:49.728497 17381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:49.728699 17381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:49.739315 17381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:49.740873 17381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:49.742265 17381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:49.756596 17386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:49.756664 17390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:49.756640 17387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:49.757543 17381 server_base.cc:1034] running on GCE node
I20260807 18:06:49.758231 17381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:49.759315 17381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:49.760610 17381 hybrid_clock.cc:648] HybridClock initialized: now 1786126009760505 us; error 70 us; skew 500 ppm
I20260807 18:06:49.760962 17381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:49 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:49.764490 17381 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:06:49.766244 17381 webserver.cc:460] Webserver started at http://127.8.62.131:45167/ using document root <none> and password file <none>
I20260807 18:06:49.766997 17381 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:49.767141 17381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:49.767479 17381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:06:49.770905 17381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "f0eb5086cb1c43d587e4e45da6374aa2"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "0f903cd6279e5efc2cd1b76d9c808438"
server_key_iv: "1d336e30c94782580faeb9740266f538"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.771783 17381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "f0eb5086cb1c43d587e4e45da6374aa2"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "0f903cd6279e5efc2cd1b76d9c808438"
server_key_iv: "1d336e30c94782580faeb9740266f538"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.777803 17381 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:06:49.781710 17399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.783138 17381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:06:49.783344 17381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "f0eb5086cb1c43d587e4e45da6374aa2"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "0f903cd6279e5efc2cd1b76d9c808438"
server_key_iv: "1d336e30c94782580faeb9740266f538"
server_key_version: "encryptionkey@0"
I20260807 18:06:49.783560 17381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:49.797775 17381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:49.803328 17381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:49.803599 17381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:49.804525 17381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:49.805953 17381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:49.806026 17381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.806119 17381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:49.806169 17381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:49.816700 17235 heartbeater.cc:503] Master 127.8.62.190:33997 was elected leader, sending a full tablet report...
I20260807 18:06:49.841799 17381 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:46691
I20260807 18:06:49.841862 17518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:46691 every 8 connection(s)
I20260807 18:06:49.843271 17381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:49.846973 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17381
I20260807 18:06:49.847196 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:06:49.847745 8442 external_mini_cluster.cc:1428] Setting key 25ba16fc0db474d606fb9d47b6aaae12
Aug 07 18:06:49 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126009, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:06:49.866816 17521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:49.846014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:46915 (local address 127.8.62.190:33997)
0807 18:06:49.846508 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:49.846514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:49.847713 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 18:06:49.850217 (+ 2504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:49.850223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:49.850380 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:49.850508 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:49.852372 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.853323 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.855195 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.855644 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.855793 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:49.859802 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:49.859807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:49.859810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:49.859848 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:49.863450 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:49.864497 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:49.864502 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:49.864504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:49.864717 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:49.865176 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:49.865179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:49.865181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:49.865458 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:49.865576 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:49.866331 (+ 755us) server_negotiation.cc:299] Negotiation successful
0807 18:06:49.866516 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":249,"threads_started":1}
I20260807 18:06:49.867861 17522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:49.846437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:46915)
0807 18:06:49.847461 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:49.847570 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:49.849927 (+ 2357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:49.850798 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:49.850814 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:49.851530 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:49.852181 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:49.852201 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.853528 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.853533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:49.855002 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:49.855024 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:49.855903 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:49.855912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:49.856187 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:49.857009 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:49.857036 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:49.859521 (+ 2485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:06:49.863644 (+ 4123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:49.863654 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:49.863794 (+ 140us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:49.864368 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:49.864830 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:06:49.864834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:06:49.864836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:06:49.865075 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:06:49.865593 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:06:49.865600 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:06:49.865971 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 18:06:49.866242 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 18:06:49.866685 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":397,"threads_started":1}
I20260807 18:06:49.869513 17519 heartbeater.cc:347] Connected to a master server at 127.8.62.190:33997
I20260807 18:06:49.869872 17519 heartbeater.cc:464] Registering TS with master...
I20260807 18:06:49.870734 17519 heartbeater.cc:511] Master 127.8.62.190:33997 requested a full tablet report, sending...
I20260807 18:06:49.872372 17036 ts_manager.cc:194] Registered new tserver with Master: f0eb5086cb1c43d587e4e45da6374aa2 (127.8.62.131:46691)
I20260807 18:06:49.873323 17036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:46915
I20260807 18:06:49.876969 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
Entry for principal oryx/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab.
WARNING: no policy specified for oryx/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
Entry for principal oryx/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab.
WARNING: no policy specified for oryx/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
Entry for principal oryx/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab.
I20260807 18:06:50.029721 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17103
Aug 07 18:06:50 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:06:50.047071 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:50.030960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41004 (local address 127.8.62.129:46845)
0807 18:06:50.031646 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:50.031655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:50.031683 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:06:50.031747 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:50.031753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:50.031970 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:50.032126 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:50.033664 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.034798 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.036076 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.036691 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.036784 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:50.041276 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:50.041287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:50.041302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:50.041341 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:50.044222 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.045056 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.045064 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.045067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.045221 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.045674 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.045678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.045680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.046033 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:50.046289 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:50.046574 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 18:06:50.046786 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":370,"threads_started":1}
I20260807 18:06:50.048664 17229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:50.133340 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17103
I20260807 18:06:50.152765 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17242
Aug 07 18:06:50 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:06:50.170518 17533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:50.153724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.8.62.130:34845)
0807 18:06:50.154322 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:50.154329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:50.154355 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:06:50.154417 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:50.154424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:50.154654 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:50.154797 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:50.156131 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.157302 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.158625 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.159189 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.159295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:50.163934 (+ 4639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:50.163949 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:50.163965 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:50.164010 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:50.167213 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.168063 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.168071 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.168074 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.168215 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.168725 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.168735 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.168738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.169154 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:50.169482 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:50.169810 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 18:06:50.170093 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":310,"threads_started":1}
I20260807 18:06:50.171841 17369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:50.256057 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17242
I20260807 18:06:50.273931 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17381
Aug 07 18:06:50 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:06:50.293107 17535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:50.274946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41134 (local address 127.8.62.131:46691)
0807 18:06:50.275505 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:50.275512 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:50.275550 (+ 38us) server_negotiation.cc:407] Connection header received
0807 18:06:50.275610 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:50.275616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:50.275828 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:50.276007 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:50.277573 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.278678 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.280055 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:50.281189 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:50.281414 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:50.286587 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:50.286603 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:50.286620 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:50.286670 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:50.290115 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.290906 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.290913 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.290916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.291059 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:50.291421 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:50.291424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:50.291426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:50.291801 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:50.292055 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:50.292385 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:06:50.292786 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":255,"threads_started":1}
I20260807 18:06:50.294615 17514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:50.379657 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17381
I20260807 18:06:50.397450 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 16998
I20260807 18:06:50.398747 17069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:06:50.503523 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 16998
I20260807 18:06:50.526209 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33997
--webserver_interface=127.8.62.190
--webserver_port=42707
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:33997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:06:50.876636 17538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:50.876976 17538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:50.877069 17538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:50.884523 17538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:06:50.884686 17538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:50.884735 17538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:50.884792 17538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:06:50.884840 17538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:06:50.895330 17538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:33997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=42707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:50.896781 17538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:50.898140 17538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:50.911540 17547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:50.911597 17545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:50.911621 17544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:50.914489 17538 server_base.cc:1034] running on GCE node
I20260807 18:06:50.915436 17538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:50.917157 17538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:50.918416 17538 hybrid_clock.cc:648] HybridClock initialized: now 1786126010918358 us; error 57 us; skew 500 ppm
I20260807 18:06:50.918836 17538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:50 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126010, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:50.922243 17538 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:06:50.923970 17538 webserver.cc:460] Webserver started at http://127.8.62.190:42707/ using document root <none> and password file <none>
I20260807 18:06:50.924772 17538 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:50.924901 17538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:50.930634 17538 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 18:06:50.933830 17555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:50.935660 17538 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:06:50.935890 17538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "f1e6bf6e19f545849fa1486ff8cb8763"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "02b0154fd1c5d956abe169b7807d563f"
server_key_iv: "db40cd1b1df12a52196673f6cef8bbfc"
server_key_version: "encryptionkey@0"
I20260807 18:06:50.937597 17538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:50.954842 17538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:50.959664 17538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:50.959918 17538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:50.980621 17538 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:33997
I20260807 18:06:50.980644 17609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:33997 every 8 connection(s)
I20260807 18:06:50.982321 17538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:06:50.989590 17610 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:06:50.990504 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17538
I20260807 18:06:50.991331 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:46845
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:33997
--webserver_port=37335
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:06:50.993242 17610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Bootstrap starting.
I20260807 18:06:51.012959 17610 log.cc:824] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Log is configured to *not* fsync() on all Append() calls
I20260807 18:06:51.022243 17610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:06:51.022795 17610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Bootstrap complete.
I20260807 18:06:51.032068 17610 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:51.033437 17610 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1e6bf6e19f545849fa1486ff8cb8763, State: Initialized, Role: FOLLOWER
I20260807 18:06:51.034253 17610 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:51.034523 17610 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:06:51.034654 17610 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:06:51.034848 17610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:06:51.038818 17610 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:51.039358 17610 leader_election.cc:302] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1e6bf6e19f545849fa1486ff8cb8763; no voters:
I20260807 18:06:51.040028 17610 leader_election.cc:288] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:06:51.040161 17614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:06:51.041610 17614 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [term 2 LEADER]: Becoming Leader. State: Replica: f1e6bf6e19f545849fa1486ff8cb8763, State: Running, Role: LEADER
I20260807 18:06:51.042253 17614 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } }
I20260807 18:06:51.042469 17610 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:06:51.045446 17616 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1e6bf6e19f545849fa1486ff8cb8763. Latest consensus state: current_term: 2 leader_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } } }
I20260807 18:06:51.045864 17616 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:51.045614 17615 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1e6bf6e19f545849fa1486ff8cb8763" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33997 } } }
I20260807 18:06:51.046056 17615 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763 [sys.catalog]: This master's current role is: LEADER
I20260807 18:06:51.058157 17623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:06:51.064242 17623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:06:51.066283 17623 catalog_manager.cc:1259] Loaded cluster ID: a54d762411694799af04aba63d0bebc2
I20260807 18:06:51.066509 17623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:06:51.072063 17623 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:06:51.074364 17623 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f1e6bf6e19f545849fa1486ff8cb8763: Loaded TSK: 0
I20260807 18:06:51.076757 17623 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:06:51.388221 17612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:51.388620 17612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:51.388795 17612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:51.396775 17612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:51.396934 17612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:51.397111 17612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:06:51.407548 17612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.129.keytab
--principal=oryx/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:46845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=37335
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:51.409093 17612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:51.410524 17612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:51.428210 17638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:51.428464 17637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:51.428723 17641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:51.431092 17612 server_base.cc:1034] running on GCE node
I20260807 18:06:51.432070 17612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:51.433537 17612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:51.434819 17612 hybrid_clock.cc:648] HybridClock initialized: now 1786126011434716 us; error 85 us; skew 500 ppm
I20260807 18:06:51.435264 17612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126011, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:51.439909 17612 init.cc:377] Logged in from keytab as oryx/127.8.62.129@KRBTEST.COM (short username oryx)
I20260807 18:06:51.441733 17612 webserver.cc:460] Webserver started at http://127.8.62.129:37335/ using document root <none> and password file <none>
I20260807 18:06:51.442574 17612 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:51.442744 17612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:51.450721 17612 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.003s sys 0.001s
I20260807 18:06:51.455011 17648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:51.457240 17612 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:06:51.457417 17612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "989db0744ede418f98f7e3944f989eec"
format_stamp: "Formatted at 2026-08-07 18:06:48 on dist-test-slave-rnrw"
server_key: "63e97cd7f6c54dfc4f420a27be2a960a"
server_key_iv: "e5c39dda10ac977bc52c9e79a3f201ed"
server_key_version: "encryptionkey@0"
I20260807 18:06:51.458570 17612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:51.487558 17612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:51.492130 17612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:51.492436 17612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:51.493547 17612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:51.495023 17612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:51.495111 17612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:51.495255 17612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:51.495347 17612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:51.530210 17612 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:46845
I20260807 18:06:51.530311 17767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:46845 every 8 connection(s)
I20260807 18:06:51.531682 17612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:06:51.537635 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17612
I20260807 18:06:51.538451 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:34845
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:33997
--webserver_port=35519
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:06:51.554749 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:51.534817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:44477 (local address 127.8.62.190:33997)
0807 18:06:51.535567 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:51.535585 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:51.535977 (+ 392us) server_negotiation.cc:407] Connection header received
0807 18:06:51.538993 (+ 3016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:51.539012 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:51.539420 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:51.539771 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:51.542075 (+ 2304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.543766 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.546572 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.547272 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.547508 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.553792 (+ 6284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:44477: BlockingRecv error: recv got EOF from 127.8.62.129:44477 (error 108)
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":234,"threads_started":1}
I20260807 18:06:51.554796 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:51.534956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:44477)
0807 18:06:51.535791 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:51.535839 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:51.538166 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:51.540013 (+ 1847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:51.540027 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:51.540917 (+ 890us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:51.541788 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.541817 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.543959 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.543967 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.546397 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.546413 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.547554 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.547562 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.547849 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.548694 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:51.548720 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:51.553372 (+ 4652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":381,"threads_started":1}
W20260807 18:06:51.555624 17768 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:33997 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.62.190:33997: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:51.565956 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:51.556345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:56059)
0807 18:06:51.556574 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:51.556597 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:51.556732 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:51.557177 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:51.557181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:51.557573 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:51.557829 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.557856 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.559266 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.559272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.560292 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.560305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.561099 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.561105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.561289 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.561972 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:51.561991 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:51.565622 (+ 3631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:06:51.566107 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:51.556602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:56059 (local address 127.8.62.190:33997)
0807 18:06:51.556818 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:51.556826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:51.556850 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:06:51.556918 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:51.556926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:51.557022 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:51.557129 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:51.558275 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.559105 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.560445 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.560936 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.561078 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.565863 (+ 4785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:56059: BlockingRecv error: recv got EOF from 127.8.62.129:56059 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:51.576624 17771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:51.566937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:40229)
0807 18:06:51.567221 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:51.567238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:51.567361 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:51.567761 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:51.567765 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:51.568187 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:51.568391 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.568402 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.569575 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.569581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.571158 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:51.571175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.571909 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.571914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:51.572140 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.572911 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:51.572937 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:51.576276 (+ 3339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162}
W20260807 18:06:51.576815 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:51.567086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:40229 (local address 127.8.62.190:33997)
0807 18:06:51.567240 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:51.567245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:51.567289 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:06:51.567551 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:51.567555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:51.567621 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:51.567708 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:51.568640 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.569407 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.571318 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:51.571766 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:51.571870 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:51.576520 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:40229: BlockingRecv error: recv got EOF from 127.8.62.129:40229 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 18:06:51.577104 17768 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:06:51.886680 17772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:51.887017 17772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:51.887189 17772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:51.894824 17772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:51.894975 17772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:51.895160 17772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:06:51.905455 17772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.130.keytab
--principal=oryx/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:34845
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=35519
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:51.906921 17772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:51.908352 17772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:51.922708 17780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:51.922765 17777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:51.922842 17778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:51.923671 17772 server_base.cc:1034] running on GCE node
I20260807 18:06:51.924393 17772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:51.925549 17772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:51.926774 17772 hybrid_clock.cc:648] HybridClock initialized: now 1786126011926719 us; error 41 us; skew 500 ppm
I20260807 18:06:51.927140 17772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:51 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126011, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:51.930574 17772 init.cc:377] Logged in from keytab as oryx/127.8.62.130@KRBTEST.COM (short username oryx)
I20260807 18:06:51.932176 17772 webserver.cc:460] Webserver started at http://127.8.62.130:35519/ using document root <none> and password file <none>
I20260807 18:06:51.933204 17772 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:51.933367 17772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:51.939090 17772 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:06:51.942049 17787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:51.943507 17772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:51.943732 17772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "85b2cf62d87648eaaf581763502a5b45"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "1bfe7dcabcf5dca58c80bd837d3f4c7f"
server_key_iv: "a6d996ca45017aabdd642e51cf9d5b7a"
server_key_version: "encryptionkey@0"
I20260807 18:06:51.944878 17772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:51.977216 17772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:51.981629 17772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:51.981925 17772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:51.982949 17772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:51.984333 17772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:51.984452 17772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:51.984603 17772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:51.984697 17772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:06:52.019137 17772 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:34845
I20260807 18:06:52.019227 17901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:34845 every 8 connection(s)
I20260807 18:06:52.020869 17772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:06:52.027383 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17772
I20260807 18:06:52.028051 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:46691
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:33997
--webserver_port=45167
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:06:52.038550 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.023474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:49683 (local address 127.8.62.190:33997)
0807 18:06:52.023699 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.023704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.025234 (+ 1530us) server_negotiation.cc:407] Connection header received
0807 18:06:52.027156 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.027161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.027259 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.027373 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.029185 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.029905 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.031534 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.031961 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.032106 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.038217 (+ 6111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:49683: BlockingRecv error: recv got EOF from 127.8.62.130:49683 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
I20260807 18:06:52.039027 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.023961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:49683)
0807 18:06:52.024949 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.025003 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.026860 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.027679 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.027689 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.028304 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.028997 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.029016 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.030092 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.030096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.031369 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.031382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.032220 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.032227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.032476 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.033294 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.033317 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.037905 (+ 4588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":287,"threads_started":1}
W20260807 18:06:52.039951 17902 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:33997 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.62.190:33997: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:52.053697 17904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.040684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:45831)
0807 18:06:52.040877 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.040900 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.041021 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.041366 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.041370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.041757 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.041979 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.041995 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.043583 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.043591 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.045491 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.045510 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.046812 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.046821 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.047071 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.047890 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.047922 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.053319 (+ 5397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:06:52.054214 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.040789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45831 (local address 127.8.62.190:33997)
0807 18:06:52.041000 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.041006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.041027 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:06:52.041168 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.041178 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.041238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.041338 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.042187 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.043421 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.045665 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.046415 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.046578 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.053927 (+ 7349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:45831: BlockingRecv error: recv got EOF from 127.8.62.130:45831 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:52.067512 17906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.054588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:44441)
0807 18:06:52.055506 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.055527 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.055739 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.056353 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.056360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.057078 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.057398 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.057413 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.059136 (+ 1723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.059142 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.060997 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.061012 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.062034 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.062042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.062223 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.062892 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.062920 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.067105 (+ 4185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":477,"threads_started":1}
W20260807 18:06:52.067631 17770 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.054710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:44441 (local address 127.8.62.190:33997)
0807 18:06:52.054928 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.054934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.055606 (+ 672us) server_negotiation.cc:407] Connection header received
0807 18:06:52.055927 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.055935 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.056032 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.056111 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.057598 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.058881 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.061194 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.061773 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.061916 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.067390 (+ 5474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:44441: BlockingRecv error: recv got EOF from 127.8.62.130:44441 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 18:06:52.068259 17902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:06:52.386123 17905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:06:52.386463 17905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:06:52.386636 17905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:06:52.394481 17905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:06:52.394613 17905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:06:52.394819 17905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:06:52.405745 17905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/krb5kdc/oryx_127.8.62.131.keytab
--principal=oryx/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:46691
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=45167
--tserver_master_addrs=127.8.62.190:33997
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:06:52.407184 17905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:06:52.408835 17905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:06:52.423323 17912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:52.423419 17914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:06:52.423367 17911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:06:52.426210 17905 server_base.cc:1034] running on GCE node
I20260807 18:06:52.427211 17905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:06:52.428345 17905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:06:52.429590 17905 hybrid_clock.cc:648] HybridClock initialized: now 1786126012429523 us; error 64 us; skew 500 ppm
I20260807 18:06:52.429960 17905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126012, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:06:52.433552 17905 init.cc:377] Logged in from keytab as oryx/127.8.62.131@KRBTEST.COM (short username oryx)
I20260807 18:06:52.435237 17905 webserver.cc:460] Webserver started at http://127.8.62.131:45167/ using document root <none> and password file <none>
I20260807 18:06:52.435995 17905 fs_manager.cc:359] Metadata directory not provided
I20260807 18:06:52.436152 17905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:06:52.442032 17905 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:06:52.445132 17921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:52.446707 17905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:06:52.446910 17905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "f0eb5086cb1c43d587e4e45da6374aa2"
format_stamp: "Formatted at 2026-08-07 18:06:49 on dist-test-slave-rnrw"
server_key: "0f903cd6279e5efc2cd1b76d9c808438"
server_key_iv: "1d336e30c94782580faeb9740266f538"
server_key_version: "encryptionkey@0"
I20260807 18:06:52.447979 17905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:06:52.461084 17905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:06:52.465581 17905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:06:52.465924 17905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:06:52.466948 17905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:06:52.468326 17905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:06:52.468441 17905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:06:52.468616 17905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:06:52.468713 17905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:06:52.502311 17905 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:46691
I20260807 18:06:52.502398 18037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:46691 every 8 connection(s)
I20260807 18:06:52.503822 17905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:06:52.505448 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 17905
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 18:06:52.522414 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.506758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:50317 (local address 127.8.62.190:33997)
0807 18:06:52.507250 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.507256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.508753 (+ 1497us) server_negotiation.cc:407] Connection header received
0807 18:06:52.510645 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.510650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.510734 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.510860 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.512330 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.513538 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.515744 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.516452 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.516624 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.522105 (+ 5481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:50317: BlockingRecv error: recv got EOF from 127.8.62.131:50317 (error 108)
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":265,"threads_started":1}
I20260807 18:06:52.523137 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.507285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:50317)
0807 18:06:52.508590 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.508647 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.510387 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.511071 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.511079 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.511597 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.512166 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.512183 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.513730 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.513735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.515599 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.515617 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.516714 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.516719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.516930 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.517766 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.517799 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.521851 (+ 4052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":421,"threads_started":1}
W20260807 18:06:52.523888 18038 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:33997 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.62.190:33997: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
I20260807 18:06:52.525563 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.506427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.8.62.190:33997)
0807 18:06:52.506627 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.506633 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.509163 (+ 2530us) server_negotiation.cc:407] Connection header received
0807 18:06:52.509374 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.509377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.509442 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.509548 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.511197 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.511900 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.513721 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.514404 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.514619 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.517831 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:06:52.517843 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:06:52.517866 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:06:52.517911 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:06:52.521255 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:52.521972 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:52.521979 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:52.521987 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:52.522181 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:06:52.522742 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:06:52.522746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:06:52.522748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:06:52.523315 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:06:52.524185 (+ 870us) server_negotiation.cc:1035] Waiting for connection context
0807 18:06:52.525025 (+ 840us) server_negotiation.cc:299] Negotiation successful
0807 18:06:52.525291 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:52.532745 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.524487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:47115)
0807 18:06:52.524678 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.524693 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.524804 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.525159 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.525164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.525486 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.525661 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.525676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.526846 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.526849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.527825 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.527836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.528674 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.528680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.528864 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.529429 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.529446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.532441 (+ 2995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:06:52.533119 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.524698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:47115 (local address 127.8.62.190:33997)
0807 18:06:52.524893 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.524899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.524916 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:06:52.524965 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.524972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.525044 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.525133 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.525829 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.526678 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.527954 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.528494 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.528658 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.532750 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:47115: BlockingRecv error: recv got EOF from 127.8.62.131:47115 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:52.541417 18042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.533476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:48353)
0807 18:06:52.533634 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.533654 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.533756 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.534231 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.534236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.534535 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.534678 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.534685 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.535653 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.535660 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.536619 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.536637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.537297 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.537302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.537475 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.537999 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.538015 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.541106 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 18:06:52.541603 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.533548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:48353 (local address 127.8.62.190:33997)
0807 18:06:52.533905 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.533909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.533922 (+ 13us) server_negotiation.cc:407] Connection header received
0807 18:06:52.534013 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.534018 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.534080 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.534182 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.534813 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.535514 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.536755 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.537159 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.537286 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.541357 (+ 4071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:48353: BlockingRecv error: recv got EOF from 127.8.62.131:48353 (error 108)
Metrics: {"server-negotiator.queue_time_us":231}
W20260807 18:06:52.541745 18038 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:52 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:52.586863 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:52.577873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:56315)
0807 18:06:52.578586 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:52.578616 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:52.578774 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:52.579128 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:52.579133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:52.579628 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:52.579831 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.579841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.580906 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.580912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.581884 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:52.581897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.582630 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.582637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:52.582814 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.583385 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:52.583402 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:52.586489 (+ 3087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":454,"threads_started":1}
W20260807 18:06:52.586977 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:52.577905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:56315 (local address 127.8.62.190:33997)
0807 18:06:52.578130 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:52.578138 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:52.578708 (+ 570us) server_negotiation.cc:407] Connection header received
0807 18:06:52.578903 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:52.578909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:52.578991 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:52.579093 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:52.579976 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.580764 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.582029 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:52.582443 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:52.582578 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:52.586744 (+ 4166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:56315: BlockingRecv error: recv got EOF from 127.8.62.129:56315 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:53.081251 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:53.069067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:47703)
0807 18:06:53.069723 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:53.069740 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:53.069912 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:53.070302 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:53.070307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:53.070829 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:53.071038 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.071048 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.072107 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.072114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.073180 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.073193 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.073995 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.074001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.074190 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.077748 (+ 3558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:53.077770 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:53.081001 (+ 3231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":336,"threads_started":1}
W20260807 18:06:53.081555 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:53.069219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:47703 (local address 127.8.62.190:33997)
0807 18:06:53.069500 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:53.069507 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:53.069842 (+ 335us) server_negotiation.cc:407] Connection header received
0807 18:06:53.070082 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:53.070086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:53.070153 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:53.070261 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:53.071210 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.071949 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.073336 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.073800 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.073954 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.081262 (+ 7308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:47703: BlockingRecv error: recv got EOF from 127.8.62.130:47703 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:53.551589 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:53.542438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:47979)
0807 18:06:53.543088 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:53.543112 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:53.543319 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:53.543750 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:53.543758 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:53.544315 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:53.544533 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.544570 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.545672 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.545676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.546680 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.546694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.547438 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.547443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.547604 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.548169 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:53.548186 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:53.551266 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":362,"threads_started":1}
W20260807 18:06:53.551759 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:53.542572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:47979 (local address 127.8.62.190:33997)
0807 18:06:53.542782 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:53.542791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:53.543244 (+ 453us) server_negotiation.cc:407] Connection header received
0807 18:06:53.543447 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:53.543452 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:53.543529 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:53.543656 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:53.544740 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.545521 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.546812 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.547269 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.547363 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.551524 (+ 4161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:47979: BlockingRecv error: recv got EOF from 127.8.62.131:47979 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:53 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:53.597587 18046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:53.588107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:55497)
0807 18:06:53.588734 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:53.588757 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:53.588918 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:53.589303 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:53.589308 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:53.589791 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:53.589944 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.589953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.591000 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.591011 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.592132 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:53.592146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.593118 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.593123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:53.593364 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.594174 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:53.594204 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:53.597336 (+ 3132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":293,"threads_started":1}
W20260807 18:06:53.598075 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:53.588194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:55497 (local address 127.8.62.190:33997)
0807 18:06:53.588464 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:53.588473 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:53.588841 (+ 368us) server_negotiation.cc:407] Connection header received
0807 18:06:53.589048 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:53.589053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:53.589141 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:53.589265 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:53.590099 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.590842 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.592281 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:53.592927 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:53.593062 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:53.597829 (+ 4767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:55497: BlockingRecv error: recv got EOF from 127.8.62.129:55497 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:54.092394 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:54.082243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:36217)
0807 18:06:54.082908 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:54.082929 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:54.083116 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:54.083524 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:54.083529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:54.084016 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:54.084181 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.084190 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.085384 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.085391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.086432 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.086446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.087280 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.087287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.087489 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.088108 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:54.088127 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:54.092028 (+ 3901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":395,"threads_started":1}
W20260807 18:06:54.092500 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:54.082340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:36217 (local address 127.8.62.190:33997)
0807 18:06:54.082655 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:54.082665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:54.083042 (+ 377us) server_negotiation.cc:407] Connection header received
0807 18:06:54.083276 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:54.083282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:54.083368 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:54.083486 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:54.084351 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.085239 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.086591 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.087114 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.087263 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.092255 (+ 4992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:36217: BlockingRecv error: recv got EOF from 127.8.62.130:36217 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:54.561811 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:54.552658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:43827)
0807 18:06:54.553218 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:54.553234 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:54.553393 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:54.553760 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:54.553764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:54.554279 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:54.554486 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.554499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.555871 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.555875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.556935 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.556948 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.557669 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.557674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.557794 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.558403 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:54.558422 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:54.561568 (+ 3146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":303,"threads_started":1}
W20260807 18:06:54.562072 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:54.552843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:43827 (local address 127.8.62.190:33997)
0807 18:06:54.553082 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:54.553088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:54.553319 (+ 231us) server_negotiation.cc:407] Connection header received
0807 18:06:54.553543 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:54.553546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:54.553622 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:54.553729 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:54.554647 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.555712 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.557077 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.557490 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.557595 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.561787 (+ 4192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:43827: BlockingRecv error: recv got EOF from 127.8.62.131:43827 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:54 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:54.607498 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:54.598638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:60467)
0807 18:06:54.599236 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:54.599256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:54.599423 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:54.599840 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:54.599845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:54.600239 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:54.600410 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.600419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.601519 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.601530 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.602453 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:54.602465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.603248 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.603257 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:54.603454 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.604221 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:54.604248 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:54.607238 (+ 2990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":325,"threads_started":1}
W20260807 18:06:54.607759 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:54.598842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:60467 (local address 127.8.62.190:33997)
0807 18:06:54.599105 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:54.599111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:54.599342 (+ 231us) server_negotiation.cc:407] Connection header received
0807 18:06:54.599602 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:54.599608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:54.599693 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:54.599801 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:54.600597 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.601358 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.602592 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:54.603062 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:54.603236 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:54.607498 (+ 4262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:60467: BlockingRecv error: recv got EOF from 127.8.62.129:60467 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:55.102680 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:55.093696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:43671)
0807 18:06:55.094304 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:55.094326 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:55.094487 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:55.094899 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:55.094903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:55.095290 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:55.095536 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.095550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.096713 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.096718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.097634 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.097645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.098347 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.098351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.098475 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.099234 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:55.099255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:55.102396 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":292,"threads_started":1}
W20260807 18:06:55.102851 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:55.093830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:43671 (local address 127.8.62.190:33997)
0807 18:06:55.094090 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:55.094096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:55.094437 (+ 341us) server_negotiation.cc:407] Connection header received
0807 18:06:55.094639 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:55.094644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:55.094755 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:55.094858 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:55.095708 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.096522 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.097773 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.098201 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.098296 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.102615 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:43671: BlockingRecv error: recv got EOF from 127.8.62.130:43671 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:55.572127 18051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:55.562821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:49663)
0807 18:06:55.563501 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:55.563520 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:55.563703 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:55.564083 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:55.564088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:55.564482 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:55.564710 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.564719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.565773 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.565780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.566730 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.566742 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.567783 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.567788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.567992 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.568668 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:55.568687 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:55.571692 (+ 3005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":346,"threads_started":1}
W20260807 18:06:55.572232 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:55.563037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:49663 (local address 127.8.62.190:33997)
0807 18:06:55.563295 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:55.563301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:55.563617 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:06:55.563865 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:55.563874 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:55.563936 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:55.564047 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:55.564881 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.565632 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.567115 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.567620 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.567767 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.571967 (+ 4200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:49663: BlockingRecv error: recv got EOF from 127.8.62.131:49663 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:55 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:55.618443 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:55.608591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:41185)
0807 18:06:55.609149 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:55.609171 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:55.609298 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:55.609671 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:55.609675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:55.610092 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:55.610306 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.610320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.611946 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.611951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.613313 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:55.613328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.614112 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.614117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:55.614268 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.614985 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:55.615005 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:55.618199 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":308,"threads_started":1}
W20260807 18:06:55.618943 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:55.608724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:41185 (local address 127.8.62.190:33997)
0807 18:06:55.609016 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:55.609023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:55.609258 (+ 235us) server_negotiation.cc:407] Connection header received
0807 18:06:55.609411 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:55.609415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:55.609485 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:55.609581 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:55.610552 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.611767 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.613460 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:55.613930 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:55.614051 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:55.618530 (+ 4479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:41185: BlockingRecv error: recv got EOF from 127.8.62.129:41185 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:56.113265 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:56.103913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:49493)
0807 18:06:56.104497 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:56.104514 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:56.104761 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:56.105148 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:56.105153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:56.105573 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:56.105722 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.105730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.106802 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.106808 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.107999 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.108013 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.108945 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.108951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.109139 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.109888 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:56.109906 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:56.112911 (+ 3005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":336,"threads_started":1}
W20260807 18:06:56.113415 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:56.104062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:49493 (local address 127.8.62.190:33997)
0807 18:06:56.104323 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:56.104330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:56.104673 (+ 343us) server_negotiation.cc:407] Connection header received
0807 18:06:56.104905 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:56.104915 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:56.104980 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:56.105126 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:56.105875 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.106653 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.108158 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.108790 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.108920 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.113176 (+ 4256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:49493: BlockingRecv error: recv got EOF from 127.8.62.130:49493 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:56.583124 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:56.573419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:49517)
0807 18:06:56.573986 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:56.574003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:56.574135 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:56.574520 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:56.574523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:56.574893 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:56.575043 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.575051 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.576239 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.576243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.577713 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.577727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.578739 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.578743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.579077 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.579769 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:56.579788 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:56.582872 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":334,"threads_started":1}
W20260807 18:06:56.583447 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:56.573607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:49517 (local address 127.8.62.190:33997)
0807 18:06:56.573800 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:56.573805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:56.574184 (+ 379us) server_negotiation.cc:407] Connection header received
0807 18:06:56.574275 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:56.574279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:56.574345 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:56.574439 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:56.575187 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.575927 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.577905 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.578546 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.578701 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.583119 (+ 4418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:49517: BlockingRecv error: recv got EOF from 127.8.62.131:49517 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:56 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:56.630095 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:56.619760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:36729)
0807 18:06:56.620441 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:56.620465 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:56.620697 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:56.621105 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:56.621110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:56.621602 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:56.621795 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.621807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.622954 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.622962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.624042 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:56.624059 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.624943 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.624950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:56.625164 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.625885 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:56.625912 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:56.629816 (+ 3904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":371,"threads_started":1}
W20260807 18:06:56.630286 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:56.619908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:36729 (local address 127.8.62.190:33997)
0807 18:06:56.620223 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:56.620230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:56.620623 (+ 393us) server_negotiation.cc:407] Connection header received
0807 18:06:56.620849 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:56.620855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:56.620935 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:56.621063 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:56.621997 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.622792 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.624220 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:56.624770 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:56.624926 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:56.630037 (+ 5111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:36729: BlockingRecv error: recv got EOF from 127.8.62.129:36729 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:57.124251 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:57.114554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:45797)
0807 18:06:57.115168 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:57.115185 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:57.115309 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:57.115696 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:57.115700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:57.116126 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:57.116299 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.116308 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.117538 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.117542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.118723 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.118738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.119702 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.119707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.119904 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.120700 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:57.120719 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:57.123876 (+ 3157us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":300,"threads_started":1}
W20260807 18:06:57.124428 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:57.114869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45797 (local address 127.8.62.190:33997)
0807 18:06:57.115104 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:57.115109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:57.115274 (+ 165us) server_negotiation.cc:407] Connection header received
0807 18:06:57.115422 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:57.115426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:57.115495 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:57.115598 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:57.116500 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.117378 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.118895 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.119475 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.119616 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.124158 (+ 4542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:45797: BlockingRecv error: recv got EOF from 127.8.62.130:45797 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:57.593116 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:57.584203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:35933)
0807 18:06:57.584820 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:57.584836 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:57.584993 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:57.585335 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:57.585339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:57.585939 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:57.586193 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.586206 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.587632 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.587636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.588525 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.588535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.589202 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.589206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.589320 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.589892 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:57.589909 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:57.592865 (+ 2956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":276,"threads_started":1}
W20260807 18:06:57.593290 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:57.584287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:35933 (local address 127.8.62.190:33997)
0807 18:06:57.584530 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:57.584537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:57.584922 (+ 385us) server_negotiation.cc:407] Connection header received
0807 18:06:57.585112 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:57.585116 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:57.585180 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:57.585297 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:57.586365 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.587492 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.588690 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.589080 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.589176 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.593074 (+ 3898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:35933: BlockingRecv error: recv got EOF from 127.8.62.131:35933 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:57 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:57.639664 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:57.631087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:40899)
0807 18:06:57.631681 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:57.631703 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:57.631869 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:57.632245 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:57.632250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:57.632781 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:57.632943 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.632952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.633993 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.634002 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.634944 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:57.634960 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.635705 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.635712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:57.635883 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.636510 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:57.636528 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:57.639322 (+ 2794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":327,"threads_started":1}
W20260807 18:06:57.639824 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:57.631175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:40899 (local address 127.8.62.190:33997)
0807 18:06:57.631509 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:57.631517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:57.631791 (+ 274us) server_negotiation.cc:407] Connection header received
0807 18:06:57.632029 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:57.632033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:57.632099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:57.632208 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:57.633097 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.633834 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.635101 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:57.635532 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:57.635669 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:57.639550 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:40899: BlockingRecv error: recv got EOF from 127.8.62.129:40899 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:58.141647 18059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:58.125503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:47183)
0807 18:06:58.126314 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:58.126330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:58.126453 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:58.126829 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:58.126833 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:58.127250 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:58.127402 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.127411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.128458 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.128463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.129510 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.129525 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.130457 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.130467 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.130770 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.137737 (+ 6967us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:58.137765 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:58.141305 (+ 3540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":561,"threads_started":1}
W20260807 18:06:58.141799 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:58.125598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:47183 (local address 127.8.62.190:33997)
0807 18:06:58.125850 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:58.125860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:58.126438 (+ 578us) server_negotiation.cc:407] Connection header received
0807 18:06:58.126570 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:58.126575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:58.126646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:58.126738 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:58.127569 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.128299 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.129678 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.130206 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.130342 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.141553 (+ 11211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:47183: BlockingRecv error: recv got EOF from 127.8.62.130:47183 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:58.602854 18060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:58.594244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:59613)
0807 18:06:58.594859 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:58.594875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:58.594992 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:58.595295 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:58.595299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:58.595683 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:58.595828 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.595835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.597020 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.597024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.598022 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.598037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.598866 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.598870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.599079 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.599692 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:58.599710 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:58.602637 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":354,"threads_started":1}
W20260807 18:06:58.603101 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:58.594362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:59613 (local address 127.8.62.190:33997)
0807 18:06:58.594550 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:58.594555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:58.594959 (+ 404us) server_negotiation.cc:407] Connection header received
0807 18:06:58.595097 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:58.595100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:58.595165 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:58.595252 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:58.595958 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.596885 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.598225 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.598688 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.598831 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.602845 (+ 4014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:59613: BlockingRecv error: recv got EOF from 127.8.62.131:59613 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:58 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:58.650182 18061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:58.640812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:50987)
0807 18:06:58.641475 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:58.641495 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:58.641624 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:58.641987 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:58.641992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:58.642450 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:58.642659 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.642669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.643972 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.643977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.644988 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:58.645001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.645676 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.645680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:58.645794 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.646456 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:58.646475 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:58.649926 (+ 3451us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":322,"threads_started":1}
W20260807 18:06:58.650353 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:58.640898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:50987 (local address 127.8.62.190:33997)
0807 18:06:58.641091 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:58.641095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:58.641565 (+ 470us) server_negotiation.cc:407] Connection header received
0807 18:06:58.641722 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:58.641726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:58.641799 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:58.641892 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:58.642799 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.643834 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.645104 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:58.645513 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:58.645610 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:58.650128 (+ 4518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:50987: BlockingRecv error: recv got EOF from 127.8.62.129:50987 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:59.152880 18062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:59.142754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:54931)
0807 18:06:59.143397 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:59.143414 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:59.143540 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:59.144056 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:59.144061 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:59.144642 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:59.144830 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.144839 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.145883 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.145889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.146853 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.146866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.147673 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.147680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.147866 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.148677 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:59.148719 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:59.152608 (+ 3889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":263,"threads_started":1}
W20260807 18:06:59.153218 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:59.142836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:54931 (local address 127.8.62.190:33997)
0807 18:06:59.143114 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:59.143124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:59.143522 (+ 398us) server_negotiation.cc:407] Connection header received
0807 18:06:59.143766 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:59.143778 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:59.143847 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:59.143999 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:59.144994 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.145733 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.147025 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.147470 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.147634 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.152922 (+ 5288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:54931: BlockingRecv error: recv got EOF from 127.8.62.130:54931 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:59.612948 18063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:59.604076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:44995)
0807 18:06:59.604779 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:59.604796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:59.604972 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:59.605354 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:59.605359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:59.605851 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:59.606032 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.606041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.607081 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.607087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.608043 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.608058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.608857 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.608864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.609037 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.609638 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:59.609656 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:59.612715 (+ 3059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":313,"threads_started":1}
W20260807 18:06:59.613170 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:59.604252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44995 (local address 127.8.62.190:33997)
0807 18:06:59.604532 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:59.604563 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:59.604910 (+ 347us) server_negotiation.cc:407] Connection header received
0807 18:06:59.605122 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:59.605127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:59.605201 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:59.605310 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:59.606189 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.606922 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.608200 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.608692 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.608833 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.612937 (+ 4104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:44995: BlockingRecv error: recv got EOF from 127.8.62.131:44995 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:06:59 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:06:59.660933 18064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:06:59.651292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:33609)
0807 18:06:59.651877 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 18:06:59.651897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:06:59.652019 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:06:59.652389 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:06:59.652393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:06:59.652867 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:06:59.653032 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.653042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.654451 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.654455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.655671 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:06:59.655685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.656577 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.656588 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:06:59.656754 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.657427 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:06:59.657446 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:06:59.660683 (+ 3237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":326,"threads_started":1}
W20260807 18:06:59.661119 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:06:59.651553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:33609 (local address 127.8.62.190:33997)
0807 18:06:59.651791 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:06:59.651797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:06:59.652015 (+ 218us) server_negotiation.cc:407] Connection header received
0807 18:06:59.652133 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:06:59.652137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:06:59.652209 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:06:59.652353 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:06:59.653227 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.654252 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.655828 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:06:59.656335 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:06:59.656475 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:06:59.660861 (+ 4386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:33609: BlockingRecv error: recv got EOF from 127.8.62.129:33609 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:00.163621 18065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:00.153828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:50739)
0807 18:07:00.154479 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:00.154497 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:00.154671 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:00.155108 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:00.155113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:00.155567 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:00.155723 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.155733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.157114 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.157119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.158080 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.158093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.158883 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.158890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.159083 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.159787 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:00.159806 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:00.163239 (+ 3433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":342,"threads_started":1}
W20260807 18:07:00.163825 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:00.153999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:50739 (local address 127.8.62.190:33997)
0807 18:07:00.154254 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:00.154262 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:00.154596 (+ 334us) server_negotiation.cc:407] Connection header received
0807 18:07:00.154836 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:00.154841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:00.154907 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:00.155014 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:00.155885 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.156946 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.158234 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.158687 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.158839 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.163542 (+ 4703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:50739: BlockingRecv error: recv got EOF from 127.8.62.130:50739 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:00.623018 18066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:00.614176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:56977)
0807 18:07:00.614785 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:00.614801 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:00.614985 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:00.615388 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:00.615392 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:00.615883 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:00.616038 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.616046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.617289 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.617296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.618196 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.618208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.618952 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.618960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.619129 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.619774 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:00.619796 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:00.622685 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":329,"threads_started":1}
W20260807 18:07:00.623172 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:00.614318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:56977 (local address 127.8.62.190:33997)
0807 18:07:00.614605 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:00.614612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:00.614898 (+ 286us) server_negotiation.cc:407] Connection header received
0807 18:07:00.615165 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:00.615169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:00.615245 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:00.615354 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:00.616192 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.617151 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.618343 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.618762 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.618890 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.622903 (+ 4013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:56977: BlockingRecv error: recv got EOF from 127.8.62.131:56977 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:00 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:00.671303 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:00.661998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:52981)
0807 18:07:00.662663 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:00.662687 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:00.662814 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:00.663196 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:00.663201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:00.663765 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:00.664027 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.664044 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.665145 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.665150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.666159 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:00.666174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.667303 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.667312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:00.667476 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.668099 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:00.668117 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:00.671048 (+ 2931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":373,"threads_started":1}
W20260807 18:07:00.671766 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:00.662072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:52981 (local address 127.8.62.190:33997)
0807 18:07:00.662268 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:00.662273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:00.662791 (+ 518us) server_negotiation.cc:407] Connection header received
0807 18:07:00.662916 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:00.662920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:00.662998 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:00.663092 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:00.664219 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.664994 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.666336 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:00.667061 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:00.667222 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:00.671295 (+ 4073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:52981: BlockingRecv error: recv got EOF from 127.8.62.129:52981 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:01.175446 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:01.164794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:60527)
0807 18:07:01.165526 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:01.165550 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:01.165723 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:01.166195 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:01.166200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:01.166922 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:01.167224 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.167237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.168732 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.168739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.169806 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.169820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.170708 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.170716 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.170928 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.171663 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:01.171683 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:01.175066 (+ 3383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":361,"threads_started":1}
W20260807 18:07:01.175608 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:01.164902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:60527 (local address 127.8.62.190:33997)
0807 18:07:01.165166 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:01.165174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:01.165652 (+ 478us) server_negotiation.cc:407] Connection header received
0807 18:07:01.165888 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:01.165894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:01.165980 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:01.166151 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:01.167397 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.168497 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.169999 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.170517 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.170680 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.175327 (+ 4647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:60527: BlockingRecv error: recv got EOF from 127.8.62.130:60527 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:01.632972 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:01.624098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:38295)
0807 18:07:01.624670 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:01.624687 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:01.624856 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:01.625224 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:01.625229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:01.625664 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:01.625825 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.625833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.626877 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.626882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.627855 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.627867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.628728 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.628733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.628924 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.629509 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:01.629527 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:01.632707 (+ 3180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":288,"threads_started":1}
W20260807 18:07:01.633054 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:01.624158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:38295 (local address 127.8.62.190:33997)
0807 18:07:01.624421 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:01.624430 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:01.624782 (+ 352us) server_negotiation.cc:407] Connection header received
0807 18:07:01.624994 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:01.624998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:01.625061 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:01.625186 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:01.626003 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.626715 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.628004 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.628468 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.628715 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.632883 (+ 4168us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:38295: BlockingRecv error: recv got EOF from 127.8.62.131:38295 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:01 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:01.680852 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:01.672238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:34351)
0807 18:07:01.672891 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:01.672916 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:01.673035 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:01.673317 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:01.673320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:01.673768 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:01.673917 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.673926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.674959 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.674963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.675969 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:01.675982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.676788 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.676796 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:01.676971 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.677635 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:01.677652 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:01.680514 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":430,"threads_started":1}
W20260807 18:07:01.681033 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:01.672344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:34351 (local address 127.8.62.190:33997)
0807 18:07:01.672533 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:01.672538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:01.673001 (+ 463us) server_negotiation.cc:407] Connection header received
0807 18:07:01.673130 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:01.673134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:01.673194 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:01.673274 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:01.674048 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.674809 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.676109 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:01.676531 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:01.676726 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:01.680790 (+ 4064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:34351: BlockingRecv error: recv got EOF from 127.8.62.129:34351 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:02.186393 18071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:02.176926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:41709)
0807 18:07:02.177492 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:02.177511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:02.177627 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:02.178040 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:02.178044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:02.178463 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:02.178656 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.178665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.179683 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.179688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.180867 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.180883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.181770 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.181780 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.182043 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.182817 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:02.182838 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:02.186134 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":327,"threads_started":1}
W20260807 18:07:02.186621 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:02.176996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:41709 (local address 127.8.62.190:33997)
0807 18:07:02.177223 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:02.177229 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:02.177601 (+ 372us) server_negotiation.cc:407] Connection header received
0807 18:07:02.177742 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:02.177746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:02.177819 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:02.177908 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:02.178809 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.179515 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.181027 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.181475 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.181574 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.186358 (+ 4784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:41709: BlockingRecv error: recv got EOF from 127.8.62.130:41709 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:02.643507 18072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:02.634091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:41751)
0807 18:07:02.634705 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:02.634723 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:02.634899 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:02.635271 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:02.635275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:02.635768 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:02.635920 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.635928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.637070 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.637075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.638041 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.638054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.638801 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.638807 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.638987 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.639721 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:02.639739 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:02.643143 (+ 3404us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":531,"threads_started":1}
W20260807 18:07:02.643687 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:02.634391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41751 (local address 127.8.62.190:33997)
0807 18:07:02.634623 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:02.634628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:02.634827 (+ 199us) server_negotiation.cc:407] Connection header received
0807 18:07:02.635060 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:02.635064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:02.635130 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:02.635236 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:02.636082 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.636922 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.638196 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.638633 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.638769 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.643360 (+ 4591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:41751: BlockingRecv error: recv got EOF from 127.8.62.131:41751 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:02 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:02.690999 18073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:02.681875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:38511)
0807 18:07:02.682407 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:02.682430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:02.682597 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:02.682916 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:02.682920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:02.683362 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:02.683610 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.683620 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.684793 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.684797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.685788 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:02.685800 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.686516 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.686520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:02.686647 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.687378 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:02.687398 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:02.690697 (+ 3299us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":306,"threads_started":1}
W20260807 18:07:02.691334 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:02.681952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:38511 (local address 127.8.62.190:33997)
0807 18:07:02.682234 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:02.682241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:02.682525 (+ 284us) server_negotiation.cc:407] Connection header received
0807 18:07:02.682712 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:02.682716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:02.682778 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:02.682863 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:02.683762 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.684655 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.685928 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:02.686365 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:02.686463 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:02.690984 (+ 4521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:38511: BlockingRecv error: recv got EOF from 127.8.62.129:38511 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:03.201318 18074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:03.187546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:55259)
0807 18:07:03.188277 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:03.188296 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:03.188500 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:03.188921 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:03.188927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:03.189424 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:03.189627 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.189637 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.190803 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.190811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.192029 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.192044 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.192932 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.192939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.193129 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.197509 (+ 4380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:03.197534 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:03.200914 (+ 3380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":418,"threads_started":1}
W20260807 18:07:03.201443 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:03.187684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:55259 (local address 127.8.62.190:33997)
0807 18:07:03.188088 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:03.188097 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:03.188406 (+ 309us) server_negotiation.cc:407] Connection header received
0807 18:07:03.188678 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:03.188690 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:03.188762 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:03.188879 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:03.189792 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.190653 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.192191 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.192757 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.192913 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.201172 (+ 8259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:55259: BlockingRecv error: recv got EOF from 127.8.62.130:55259 (error 108)
Metrics: {"server-negotiator.queue_time_us":208}
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:03.654698 18075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:03.644803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:44295)
0807 18:07:03.645612 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:03.645630 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:03.645756 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:03.646071 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:03.646075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:03.646501 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:03.646747 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.646757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.647784 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.647788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.648918 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.648932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.649617 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.649622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.649755 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.650508 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:03.650529 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:03.654348 (+ 3819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":551,"threads_started":1}
W20260807 18:07:03.654901 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:03.644880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44295 (local address 127.8.62.190:33997)
0807 18:07:03.645162 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:03.645173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:03.645717 (+ 544us) server_negotiation.cc:407] Connection header received
0807 18:07:03.645863 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:03.645867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:03.645934 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:03.646024 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:03.646877 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.647636 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.649066 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.649468 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.649599 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.654673 (+ 5074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:44295: BlockingRecv error: recv got EOF from 127.8.62.131:44295 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:03 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:03.701265 18076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:03.692128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:57275)
0807 18:07:03.692716 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:03.692749 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:03.692867 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:03.693241 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:03.693244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:03.693653 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:03.693820 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.693829 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.694883 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.694887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.695833 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:03.695846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.696691 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.696696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:03.696830 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.697588 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:03.697607 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:03.700888 (+ 3281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":312,"threads_started":1}
W20260807 18:07:03.701454 18041 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:03.692187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:57275 (local address 127.8.62.190:33997)
0807 18:07:03.692415 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:03.692420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:03.692839 (+ 419us) server_negotiation.cc:407] Connection header received
0807 18:07:03.693015 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:03.693019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:03.693086 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:03.693176 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:03.693973 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.694748 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.695996 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:03.696471 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:03.696678 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:03.701149 (+ 4471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:57275: BlockingRecv error: recv got EOF from 127.8.62.129:57275 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:04.211992 18077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:04.202547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:57895)
0807 18:07:04.203204 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:04.203227 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:04.203377 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:04.203811 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:04.203814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:04.204228 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:04.204404 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.204412 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.205690 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.205695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.206681 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.206693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.207468 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.207473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.207615 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.208238 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:04.208258 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:04.211635 (+ 3377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":308,"threads_started":1}
W20260807 18:07:04.212225 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:04.202618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:57895 (local address 127.8.62.190:33997)
0807 18:07:04.203359 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:04.203365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:04.203383 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:07:04.203489 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:04.203493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:04.203604 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:04.203708 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:04.204655 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.205518 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.206828 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.207295 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.207409 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.211924 (+ 4515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:57895: BlockingRecv error: recv got EOF from 127.8.62.130:57895 (error 108)
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":545,"threads_started":1}
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:04.664743 18079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:04.655782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:43443)
0807 18:07:04.656409 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:04.656425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:04.656586 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:04.656904 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:04.656907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:04.657302 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:04.657464 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.657473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.658501 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.658505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.659494 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.659508 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.660224 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.660228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.660371 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.661104 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:04.661125 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:04.664365 (+ 3240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":355,"threads_started":1}
W20260807 18:07:04.664903 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:04.655830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:43443 (local address 127.8.62.190:33997)
0807 18:07:04.656001 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:04.656005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:04.656509 (+ 504us) server_negotiation.cc:407] Connection header received
0807 18:07:04.656697 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:04.656701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:04.656765 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:04.656849 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:04.657598 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.658349 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.659641 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.660064 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.660166 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.664652 (+ 4486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:43443: BlockingRecv error: recv got EOF from 127.8.62.131:43443 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:04 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:04.711442 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:04.702488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:37503)
0807 18:07:04.703344 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:04.703365 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:04.703542 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:04.703956 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:04.703961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:04.704531 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:04.704839 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.704855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.705928 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.705932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.706858 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:04.706869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.707676 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.707680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:04.707798 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.708378 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:04.708395 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:04.711202 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":569,"threads_started":1}
W20260807 18:07:04.711700 18078 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:04.702676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:37503 (local address 127.8.62.190:33997)
0807 18:07:04.702933 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:04.702940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:04.703473 (+ 533us) server_negotiation.cc:407] Connection header received
0807 18:07:04.703688 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:04.703692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:04.703788 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:04.703887 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:04.705006 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.705796 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.707013 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:04.707505 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:04.707614 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:04.711409 (+ 3795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:37503: BlockingRecv error: recv got EOF from 127.8.62.129:37503 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:05.222520 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:05.213307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:46031)
0807 18:07:05.213893 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:05.213910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:05.214022 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:05.214456 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:05.214462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:05.214899 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:05.215126 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.215137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.216201 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.216205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.217261 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.217275 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.218025 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.218029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.218179 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.218905 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:05.218945 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:05.222288 (+ 3343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":279,"threads_started":1}
W20260807 18:07:05.222862 18082 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:05.213475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:46031 (local address 127.8.62.190:33997)
0807 18:07:05.213962 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:05.213967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:05.213986 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:07:05.214179 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:05.214183 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:05.214259 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:05.214358 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:05.215290 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.216040 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.217417 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.217860 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.217965 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.222508 (+ 4543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:46031: BlockingRecv error: recv got EOF from 127.8.62.130:46031 (error 108)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":283,"threads_started":1}
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:05.675292 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:05.665949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:39973)
0807 18:07:05.666563 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:05.666579 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:05.666754 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:05.667203 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:05.667209 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:05.667797 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:05.667995 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.668008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.669402 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.669406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.670371 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.670384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.671151 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.671155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.671281 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.671892 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:05.671914 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:05.675042 (+ 3128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":302,"threads_started":1}
W20260807 18:07:05.675556 18082 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:05.666100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:39973 (local address 127.8.62.190:33997)
0807 18:07:05.666347 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:05.666354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:05.666678 (+ 324us) server_negotiation.cc:407] Connection header received
0807 18:07:05.666891 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:05.666896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:05.666983 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:05.667112 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:05.668220 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.669266 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.670553 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.670968 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.671079 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.675282 (+ 4203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:39973: BlockingRecv error: recv got EOF from 127.8.62.131:39973 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:05 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:05.721748 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:05.712590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:53469)
0807 18:07:05.713111 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:05.713138 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:05.713269 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:05.713624 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:05.713627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:05.713998 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:05.714178 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.714185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.715287 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.715292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.716402 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:05.716416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.717445 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.717452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:05.717632 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.718450 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:05.718468 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:05.721479 (+ 3011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":291,"threads_started":1}
W20260807 18:07:05.722128 18082 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:05.712652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:53469 (local address 127.8.62.190:33997)
0807 18:07:05.712923 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:05.712927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:05.713232 (+ 305us) server_negotiation.cc:407] Connection header received
0807 18:07:05.713371 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:05.713375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:05.713452 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:05.713541 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:05.714316 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.715124 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.716638 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:05.717267 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:05.717364 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:05.721729 (+ 4365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:53469: BlockingRecv error: recv got EOF from 127.8.62.129:53469 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:06.232757 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:06.223596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:39505)
0807 18:07:06.224122 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:06.224139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:06.224251 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:06.224664 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:06.224668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:06.225056 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:06.225229 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.225239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.226303 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.226307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.227412 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.227441 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.228464 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.228471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.228697 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.229349 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:06.229367 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:06.232381 (+ 3014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":275,"threads_started":1}
W20260807 18:07:06.232911 18086 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:06.223740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:39505 (local address 127.8.62.190:33997)
0807 18:07:06.224215 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:06.224219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:06.224238 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:07:06.224370 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:06.224374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:06.224460 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:06.224642 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:06.225377 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.226137 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.227587 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.228271 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.228419 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.232653 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:39505: BlockingRecv error: recv got EOF from 127.8.62.130:39505 (error 108)
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":279,"threads_started":1}
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.131@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:06.684741 18087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:06.676361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.131:33265)
0807 18:07:06.676991 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:06.677008 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:06.677142 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:06.677446 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:06.677449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:06.677816 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:06.677996 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.678004 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.678989 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.678993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.679942 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.679954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.680709 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.680714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.680836 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.681494 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:06.681511 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:06.684406 (+ 2895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":379,"threads_started":1}
W20260807 18:07:06.684907 18086 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:06.676502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:33265 (local address 127.8.62.190:33997)
0807 18:07:06.676718 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:06.676723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:06.677104 (+ 381us) server_negotiation.cc:407] Connection header received
0807 18:07:06.677245 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:06.677248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:06.677312 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:06.677398 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:06.678125 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.678858 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.680104 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.680522 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.680660 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.684654 (+ 3994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.131:33265: BlockingRecv error: recv got EOF from 127.8.62.131:33265 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:06 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.129@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:06.731407 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:06.722935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.129:39869)
0807 18:07:06.723402 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:06.723422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:06.723536 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:06.723862 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:06.723865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:06.724278 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:06.724466 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.724474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.725715 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.725719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.726671 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:06.726683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.727308 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.727312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:06.727443 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.728065 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:06.728083 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:06.731079 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":267,"threads_started":1}
W20260807 18:07:06.731645 18086 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:06.723014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:39869 (local address 127.8.62.190:33997)
0807 18:07:06.723273 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:06.723279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:06.723505 (+ 226us) server_negotiation.cc:407] Connection header received
0807 18:07:06.723645 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:06.723648 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:06.723723 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:06.723806 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:06.724770 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.725567 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.726793 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:06.727180 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:06.727272 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:06.731379 (+ 4107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.129:39869: BlockingRecv error: recv got EOF from 127.8.62.129:39869 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 18:07:07 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 18:07:07 dist-test-slave-rnrw krb5kdc[16985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.62.130@KRBTEST.COM for oryx/127.8.62.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 18:07:07.243466 18089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:07.234057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:33997 (local address 127.8.62.130:51369)
0807 18:07:07.234652 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:07.234670 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:07.234779 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:07.235297 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:07.235301 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:07.235788 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:07.235994 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:07.236004 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.237610 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.237615 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:07.238551 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:07.238563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.239335 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.239341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:07.239526 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:07.240114 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:07.240133 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:07.243208 (+ 3075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.62.190:33997: Server oryx/127.8.62.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":352,"threads_started":1}
W20260807 18:07:07.243924 18090 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:07.234198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:51369 (local address 127.8.62.190:33997)
0807 18:07:07.234741 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:07.234746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:07.234765 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:07:07.234955 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:07.234960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:07.235048 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:07.235180 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:07.236245 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.237428 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.238713 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.239173 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.239318 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:07.243637 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.62.130:51369: BlockingRecv error: recv got EOF from 127.8.62.130:51369 (error 108)
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":308,"threads_started":1}
I20260807 18:07:07.509859 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17612
I20260807 18:07:07.522807 18092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:07.511026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.8.62.129:46845)
0807 18:07:07.511667 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:07.511674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:07.511700 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:07:07.511763 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:07.511769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:07.511984 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:07.512159 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:07.513843 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.514977 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.516286 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.516976 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.517091 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:07.519674 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:07.519682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:07.519697 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:07.519735 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:07.522310 (+ 2575us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:07:07.522574 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49180: Request ticket server kudu/127.8.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":350,"threads_started":1}
W20260807 18:07:07.522933 18092 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49180: Request ticket server kudu/127.8.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:07:07.522965 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17612: Not authorized: Client connection negotiation failed: client connection to 127.8.62.129:46845: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:07:07.523240 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17612
I20260807 18:07:07.541777 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17772
I20260807 18:07:07.556154 18093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:07.543106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55520 (local address 127.8.62.130:34845)
0807 18:07:07.543845 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:07.543853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:07.543881 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:07:07.543947 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:07.543953 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:07.544212 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:07.544400 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:07.545956 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.547076 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.548443 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.549280 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.549388 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:07.552739 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:07.552749 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:07.552765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:07.552832 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:07.555536 (+ 2704us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:07:07.555842 (+ 306us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55520: Request ticket server kudu/127.8.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":390,"threads_started":1}
W20260807 18:07:07.556278 18093 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55520: Request ticket server kudu/127.8.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:07:07.556386 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17772: Not authorized: Client connection negotiation failed: client connection to 127.8.62.130:34845: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:07:07.556684 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17772
I20260807 18:07:07.575129 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17905
I20260807 18:07:07.589097 18094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:07.576265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.8.62.131:46691)
0807 18:07:07.577055 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:07.577062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:07.577090 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:07:07.577156 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:07.577162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:07.577391 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:07.577547 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:07.578929 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.580063 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.581637 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:07.582205 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:07.582311 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:07.585235 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:07.585244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:07.585258 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:07.585298 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:07.588510 (+ 3212us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 18:07:07.588852 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43846: Request ticket server kudu/127.8.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":433,"threads_started":1}
W20260807 18:07:07.589229 18094 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43846: Request ticket server kudu/127.8.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 18:07:07.589366 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17905: Not authorized: Client connection negotiation failed: client connection to 127.8.62.131:46691: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.62.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 18:07:07.589586 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17905
I20260807 18:07:07.607285 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 17538
I20260807 18:07:07.608855 17605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:07.728302 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 17538
2026-08-07T18:07:07Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21441 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T18:07:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:07:07Z Disabled control of system clock
I20260807 18:07:07.812120 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:39475
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:39475
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 18:07:08.162489 18100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:08.162837 18100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:08.162921 18100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:08.170642 18100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:07:08.170785 18100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:08.170825 18100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:08.170877 18100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:07:08.170928 18100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:07:08.180615 18100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:39475
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:39475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:08.182022 18100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:08.183794 18100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:08.197456 18105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:08.197504 18106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:08.197410 18108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:08.198140 18100 server_base.cc:1034] running on GCE node
I20260807 18:07:08.198973 18100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:08.200479 18100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:08.201759 18100 hybrid_clock.cc:648] HybridClock initialized: now 1786126028201713 us; error 42 us; skew 500 ppm
I20260807 18:07:08.202188 18100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:08.209354 18100 webserver.cc:460] Webserver started at http://127.8.62.190:39945/ using document root <none> and password file <none>
I20260807 18:07:08.210129 18100 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:08.210269 18100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:08.210640 18100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:08.213747 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "6c0046e77b63479e8a607f1079668057"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "3df27315ec43ec35b73ffbfab6b6b00d"
server_key_iv: "072cc50aced3997351f9ff72dd672a81"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.214591 18100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "6c0046e77b63479e8a607f1079668057"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "3df27315ec43ec35b73ffbfab6b6b00d"
server_key_iv: "072cc50aced3997351f9ff72dd672a81"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.220103 18100 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 18:07:08.223804 18114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:08.225301 18100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:07:08.225500 18100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "6c0046e77b63479e8a607f1079668057"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "3df27315ec43ec35b73ffbfab6b6b00d"
server_key_iv: "072cc50aced3997351f9ff72dd672a81"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.225733 18100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:08.263079 18100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:08.264062 18100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:08.264385 18100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:08.284229 18100 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:39475
I20260807 18:07:08.284248 18167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:39475 every 8 connection(s)
I20260807 18:07:08.285764 18100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:07:08.287727 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18100
I20260807 18:07:08.287930 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:07:08.288779 8442 external_mini_cluster.cc:1428] Setting key 17d8593fc669c61f9d15d1d09c9c9a27
I20260807 18:07:08.291142 18168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:08.305124 18171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:08.290977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48004 (local address 127.8.62.190:39475)
0807 18:07:08.292095 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:08.292113 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:08.292167 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:07:08.294393 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:08.294424 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:08.295096 (+ 672us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:08.295559 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:08.296113 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.298221 (+ 2108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.299557 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.300687 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.300870 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:08.302134 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:08.302147 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:08.302166 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:08.302974 (+ 808us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:08.303085 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:08.303117 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:08.303235 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:08.303472 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:07:08.303843 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":401,"threads_started":1}
I20260807 18:07:08.306095 18168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057: Bootstrap starting.
I20260807 18:07:08.309389 18168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:08.310441 18168 log.cc:824] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057: Log is configured to *not* fsync() on all Append() calls
I20260807 18:07:08.313549 18168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057: No bootstrap required, opened a new log
I20260807 18:07:08.320941 18168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } }
I20260807 18:07:08.321355 18168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:08.321476 18168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c0046e77b63479e8a607f1079668057, State: Initialized, Role: FOLLOWER
I20260807 18:07:08.322079 18168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } }
I20260807 18:07:08.322331 18168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:07:08.322486 18168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:07:08.322672 18168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:08.325085 18168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } }
I20260807 18:07:08.325639 18168 leader_election.cc:302] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c0046e77b63479e8a607f1079668057; no voters:
I20260807 18:07:08.326260 18168 leader_election.cc:288] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:07:08.326401 18173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:07:08.327109 18173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [term 1 LEADER]: Becoming Leader. State: Replica: 6c0046e77b63479e8a607f1079668057, State: Running, Role: LEADER
I20260807 18:07:08.327662 18173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } }
I20260807 18:07:08.328466 18168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:07:08.331012 18174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c0046e77b63479e8a607f1079668057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } } }
I20260807 18:07:08.330988 18175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c0046e77b63479e8a607f1079668057. Latest consensus state: current_term: 1 leader_uuid: "6c0046e77b63479e8a607f1079668057" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c0046e77b63479e8a607f1079668057" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39475 } } }
I20260807 18:07:08.331485 18175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:08.336429 18174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:08.338940 18179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:07:08.345461 18179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:07:08.355085 18179 catalog_manager.cc:1347] Generated new cluster ID: 950f8129697249d59f3dd80dacaeb3d7
I20260807 18:07:08.355233 18179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:07:08.372323 18179 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:07:08.373958 18179 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:07:08.381664 18179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c0046e77b63479e8a607f1079668057: Generated new TSK 0
I20260807 18:07:08.382506 18179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:07:08.389956 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--builtin_ntp_servers=127.8.62.148:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 18:07:08.726675 18193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:08.727018 18193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:08.727188 18193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:08.735004 18193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:08.735169 18193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:08.735359 18193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:07:08.745110 18193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:08.746460 18193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:08.748320 18193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:08.761714 18198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:08.761752 18199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:08.761682 18201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:08.762609 18193 server_base.cc:1034] running on GCE node
I20260807 18:07:08.763257 18193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:08.764267 18193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:08.765465 18193 hybrid_clock.cc:648] HybridClock initialized: now 1786126028765397 us; error 57 us; skew 500 ppm
I20260807 18:07:08.765838 18193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:08.768352 18193 webserver.cc:460] Webserver started at http://127.8.62.129:39893/ using document root <none> and password file <none>
I20260807 18:07:08.769192 18193 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:08.769289 18193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:08.769647 18193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:08.772512 18193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "605a5c2f97764215b93423db7778b220"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "7310b00cb09d9e3b5ca162ebe78c7142"
server_key_iv: "6f46a2c056c468f6c2852a826a44a4e5"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.773397 18193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "605a5c2f97764215b93423db7778b220"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "7310b00cb09d9e3b5ca162ebe78c7142"
server_key_iv: "6f46a2c056c468f6c2852a826a44a4e5"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.779443 18193 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:07:08.784729 18207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:08.786374 18193 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:07:08.786566 18193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "605a5c2f97764215b93423db7778b220"
format_stamp: "Formatted at 2026-08-07 18:07:08 on dist-test-slave-rnrw"
server_key: "7310b00cb09d9e3b5ca162ebe78c7142"
server_key_iv: "6f46a2c056c468f6c2852a826a44a4e5"
server_key_version: "encryptionkey@0"
I20260807 18:07:08.786805 18193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:08.803628 18193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:08.804589 18193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:08.804915 18193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:08.805895 18193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:08.807551 18193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:08.807693 18193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:08.807839 18193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:08.807945 18193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:08.843515 18193 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:36623
I20260807 18:07:08.843639 18325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:36623 every 8 connection(s)
I20260807 18:07:08.845057 18193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:07:08.854939 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18193
I20260807 18:07:08.855168 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:07:08.855821 8442 external_mini_cluster.cc:1428] Setting key 593a9a269ab7b411768b48c1cda65b68
I20260807 18:07:08.858664 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:08.847824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:45953 (local address 127.8.62.190:39475)
0807 18:07:08.848280 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:08.848285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:08.849579 (+ 1294us) server_negotiation.cc:407] Connection header received
0807 18:07:08.851505 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:08.851511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:08.851615 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:08.851737 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:08.852942 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.853696 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.855498 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.855944 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.856103 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:08.857741 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:08.857746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:08.857750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:08.857769 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:08.857837 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:08.857852 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:08.857948 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:08.858204 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 18:07:08.858307 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260807 18:07:08.859016 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--builtin_ntp_servers=127.8.62.148:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:07:08.859740 18329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:08.848451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:39475 (local address 127.8.62.129:45953)
0807 18:07:08.849401 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:08.849447 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:08.851150 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:08.851991 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:08.852000 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:08.852095 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:08.852762 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:08.852783 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.853849 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.853857 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:08.855359 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:08.855371 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:08.856200 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:08.856210 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:08.856453 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:08.857360 (+ 907us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:08.857363 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:08.857404 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:08.857487 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:08.857961 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:08.857971 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:08.857977 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:07:08.858135 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:07:08.858461 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":263,"threads_started":1}
I20260807 18:07:08.861467 18326 heartbeater.cc:347] Connected to a master server at 127.8.62.190:39475
I20260807 18:07:08.861819 18326 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:08.862838 18326 heartbeater.cc:511] Master 127.8.62.190:39475 requested a full tablet report, sending...
I20260807 18:07:08.865476 18133 ts_manager.cc:194] Registered new tserver with Master: 605a5c2f97764215b93423db7778b220 (127.8.62.129:36623)
I20260807 18:07:08.867226 18133 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.129:45953
W20260807 18:07:09.200831 18330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:09.201190 18330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:09.201355 18330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:09.209025 18330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:09.209165 18330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:09.209347 18330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:07:09.218854 18330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:09.220250 18330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:09.222137 18330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:09.236470 18335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:09.236801 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:09.236825 18338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:09.237532 18330 server_base.cc:1034] running on GCE node
I20260807 18:07:09.238305 18330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:09.239491 18330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:09.240746 18330 hybrid_clock.cc:648] HybridClock initialized: now 1786126029240682 us; error 46 us; skew 500 ppm
I20260807 18:07:09.241169 18330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:09.244103 18330 webserver.cc:460] Webserver started at http://127.8.62.130:38689/ using document root <none> and password file <none>
I20260807 18:07:09.245008 18330 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:09.245139 18330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:09.245524 18330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:09.248715 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "17d98ea073c24d6aa294cb0177f90f7e"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "51b3cf704e09e6524e841e4ce762d4e9"
server_key_iv: "ec0d328766a03400dc3c8feb0b026a13"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.249564 18330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "17d98ea073c24d6aa294cb0177f90f7e"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "51b3cf704e09e6524e841e4ce762d4e9"
server_key_iv: "ec0d328766a03400dc3c8feb0b026a13"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.255240 18330 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:07:09.259699 18345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.261226 18330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:07:09.261423 18330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "17d98ea073c24d6aa294cb0177f90f7e"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "51b3cf704e09e6524e841e4ce762d4e9"
server_key_iv: "ec0d328766a03400dc3c8feb0b026a13"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.261682 18330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:09.289444 18330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:09.290464 18330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:09.290832 18330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:09.292050 18330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:09.293741 18330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:09.293875 18330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.294025 18330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:09.294222 18330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.330797 18330 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:45895
I20260807 18:07:09.330893 18461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:45895 every 8 connection(s)
I20260807 18:07:09.332252 18330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:07:09.338258 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18330
I20260807 18:07:09.338443 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:07:09.339033 8442 external_mini_cluster.cc:1428] Setting key 7b99e55a6423cc7864ae3466cd48fec3
I20260807 18:07:09.342239 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--builtin_ntp_servers=127.8.62.148:37911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 18:07:09.347011 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.335287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:35647 (local address 127.8.62.190:39475)
0807 18:07:09.335492 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:09.335497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:09.337140 (+ 1643us) server_negotiation.cc:407] Connection header received
0807 18:07:09.339795 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:09.339802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:09.339910 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:09.340018 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:09.341547 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.342277 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.343975 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.344377 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.344589 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.346107 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:09.346114 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:09.346123 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:09.346146 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:09.346179 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:09.346192 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:09.346308 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:09.346553 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 18:07:09.346673 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:07:09.348227 18464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.335876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:39475 (local address 127.8.62.130:35647)
0807 18:07:09.336956 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:09.336999 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:09.339329 (+ 2330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:09.340334 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:09.340344 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:09.340453 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:09.341327 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:09.341354 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.342465 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.342473 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:09.343812 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:09.343825 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.344746 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.344761 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:09.344987 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.345755 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:09.345758 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:09.345801 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:09.345892 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:09.346297 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:09.346309 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:09.346319 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:07:09.346526 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 18:07:09.346976 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":311,"threads_started":1}
I20260807 18:07:09.349916 18462 heartbeater.cc:347] Connected to a master server at 127.8.62.190:39475
I20260807 18:07:09.350296 18462 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:09.351123 18462 heartbeater.cc:511] Master 127.8.62.190:39475 requested a full tablet report, sending...
I20260807 18:07:09.352676 18133 ts_manager.cc:194] Registered new tserver with Master: 17d98ea073c24d6aa294cb0177f90f7e (127.8.62.130:45895)
I20260807 18:07:09.353530 18133 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.130:35647
W20260807 18:07:09.681424 18465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:09.681800 18465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:09.681969 18465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:09.689499 18465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:09.689667 18465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:09.689847 18465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:07:09.699496 18465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:39475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:09.700941 18465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:09.702893 18465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:09.716964 18470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:09.716948 18471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:09.717519 18473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:09.717803 18465 server_base.cc:1034] running on GCE node
I20260807 18:07:09.718420 18465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:09.719408 18465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:09.720649 18465 hybrid_clock.cc:648] HybridClock initialized: now 1786126029720605 us; error 46 us; skew 500 ppm
I20260807 18:07:09.720940 18465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:09.723245 18465 webserver.cc:460] Webserver started at http://127.8.62.131:36113/ using document root <none> and password file <none>
I20260807 18:07:09.723927 18465 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:09.724015 18465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:09.724289 18465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:09.727317 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "3ee5edacf60749dfa15a5ff652218d52"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "052a533953ba6a1a58ce9766bd0c5943"
server_key_iv: "f185aca6b0e103432f6713c627f14499"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.728091 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "3ee5edacf60749dfa15a5ff652218d52"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "052a533953ba6a1a58ce9766bd0c5943"
server_key_iv: "f185aca6b0e103432f6713c627f14499"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.733595 18465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 18:07:09.737308 18479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.738860 18465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:07:09.739004 18465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "3ee5edacf60749dfa15a5ff652218d52"
format_stamp: "Formatted at 2026-08-07 18:07:09 on dist-test-slave-rnrw"
server_key: "052a533953ba6a1a58ce9766bd0c5943"
server_key_iv: "f185aca6b0e103432f6713c627f14499"
server_key_version: "encryptionkey@0"
I20260807 18:07:09.739277 18465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:09.751667 18465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:09.752535 18465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:09.752925 18465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:09.753942 18465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:09.755439 18465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:09.755528 18465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.755693 18465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:09.755780 18465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:09.790215 18465 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:42987
I20260807 18:07:09.790346 18595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:42987 every 8 connection(s)
I20260807 18:07:09.791888 18465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:07:09.798143 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18465
I20260807 18:07:09.798336 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:07:09.798928 8442 external_mini_cluster.cc:1428] Setting key 2f0079137990403072e4bd4c97267369
I20260807 18:07:09.805413 18328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.794580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:32807 (local address 127.8.62.190:39475)
0807 18:07:09.794774 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:09.794779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:09.796139 (+ 1360us) server_negotiation.cc:407] Connection header received
0807 18:07:09.798227 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:09.798264 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:09.798436 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:09.798633 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:09.800008 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.801007 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.802934 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.803338 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.803432 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.804765 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:09.804769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:09.804773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:09.804791 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:09.804820 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:09.804832 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:09.804903 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:09.805105 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 18:07:09.805183 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:07:09.806430 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.795135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:39475 (local address 127.8.62.131:32807)
0807 18:07:09.796005 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:09.796046 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:09.797834 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:09.798938 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:09.798952 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:09.799087 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:09.799814 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:09.799832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.801212 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.801219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:09.802802 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:09.802815 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.803519 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.803524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:09.803716 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.804413 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:09.804415 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:09.804459 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:09.804536 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:09.804914 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:09.804921 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:09.804926 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 18:07:09.805073 (+ 147us) client_negotiation.cc:240] Negotiation successful
0807 18:07:09.805370 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":261,"threads_started":1}
I20260807 18:07:09.807844 18596 heartbeater.cc:347] Connected to a master server at 127.8.62.190:39475
I20260807 18:07:09.808144 18596 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:09.808945 18596 heartbeater.cc:511] Master 127.8.62.190:39475 requested a full tablet report, sending...
I20260807 18:07:09.810427 18133 ts_manager.cc:194] Registered new tserver with Master: 3ee5edacf60749dfa15a5ff652218d52 (127.8.62.131:42987)
I20260807 18:07:09.811173 18133 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.131:32807
I20260807 18:07:09.814606 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:07:09.821121 18328 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:09.819184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33288 (local address 127.8.62.190:39475)
0807 18:07:09.819389 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:09.819394 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:09.819923 (+ 529us) server_negotiation.cc:407] Connection header received
0807 18:07:09.820103 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:09.820107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:09.820187 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:09.820279 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:09.820849 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33288: BlockingRecv error: recv got EOF from 127.0.0.1:33288 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:07:09.821703 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18193
I20260807 18:07:09.829694 18605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.822714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.8.62.129:36623)
0807 18:07:09.823266 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:09.823273 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:09.823298 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:07:09.823359 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:09.823365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:09.823569 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:09.823703 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:09.824048 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.825105 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.826318 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.826870 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.826964 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.828010 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:09.828017 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:09.828030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:09.828789 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:09.828842 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:09.828862 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:09.828948 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:09.829102 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 18:07:09.829255 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":286,"threads_started":1}
I20260807 18:07:09.831017 18321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:09.914680 18326 heartbeater.cc:503] Master 127.8.62.190:39475 was elected leader, sending a full tablet report...
I20260807 18:07:09.915894 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18193
W20260807 18:07:09.933702 18118 connection.cc:535] server connection from 127.8.62.129:45953 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:07:09.934067 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18330
I20260807 18:07:09.942291 18607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:09.934981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.8.62.130:45895)
0807 18:07:09.935486 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:09.935493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:09.935520 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:07:09.935577 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:09.935583 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:09.935807 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:09.935933 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:09.936333 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.937458 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.938919 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:09.939537 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:09.939637 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:09.940848 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:09.940856 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:09.940869 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:09.941462 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:09.941508 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:09.941526 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:09.941628 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:09.941767 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:07:09.941898 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260807 18:07:09.943734 18457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:10.026777 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18330
I20260807 18:07:10.045249 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18465
I20260807 18:07:10.054168 18609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:10.046224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39760 (local address 127.8.62.131:42987)
0807 18:07:10.046960 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:10.046968 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:10.046999 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:07:10.047061 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:10.047068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:10.047307 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:10.047462 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:10.047878 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:10.049119 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:10.050497 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:10.051119 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:10.051240 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:10.052287 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:10.052296 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:10.052312 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:10.053156 (+ 844us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:10.053206 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:10.053227 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:10.053327 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:10.053574 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:07:10.053812 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":421,"threads_started":1}
I20260807 18:07:10.055543 18591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:10.141036 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18465
I20260807 18:07:10.161238 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18100
I20260807 18:07:10.162530 18163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:10.269109 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18100
2026-08-07T18:07:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2534 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T18:07:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:07:10Z Disabled control of system clock
I20260807 18:07:10.347934 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38229
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:38229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:07:10.697095 18615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:10.697407 18615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:10.697492 18615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:10.704824 18615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:07:10.704953 18615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:10.704991 18615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:10.705042 18615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:07:10.705080 18615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:07:10.714582 18615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:38229
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:38229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:10.715885 18615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:10.717752 18615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:10.729952 18620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:10.730020 18621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:10.730044 18623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:10.731109 18615 server_base.cc:1034] running on GCE node
I20260807 18:07:10.732015 18615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:10.733549 18615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:10.734808 18615 hybrid_clock.cc:648] HybridClock initialized: now 1786126030734755 us; error 33 us; skew 500 ppm
I20260807 18:07:10.735299 18615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:10.737920 18615 webserver.cc:460] Webserver started at http://127.8.62.190:41871/ using document root <none> and password file <none>
I20260807 18:07:10.738759 18615 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:10.738893 18615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:10.739279 18615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:10.742764 18615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "3b85523a76434f6f9b19081d7e5176e3"
format_stamp: "Formatted at 2026-08-07 18:07:10 on dist-test-slave-rnrw"
server_key: "d5c31802313edb32e69a1375c58daa91"
server_key_iv: "cf1f8977ee1247b3e2617365cd9618a6"
server_key_version: "encryptionkey@0"
I20260807 18:07:10.743731 18615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "3b85523a76434f6f9b19081d7e5176e3"
format_stamp: "Formatted at 2026-08-07 18:07:10 on dist-test-slave-rnrw"
server_key: "d5c31802313edb32e69a1375c58daa91"
server_key_iv: "cf1f8977ee1247b3e2617365cd9618a6"
server_key_version: "encryptionkey@0"
I20260807 18:07:10.749673 18615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:07:10.753674 18629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:10.755115 18615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:07:10.755306 18615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "3b85523a76434f6f9b19081d7e5176e3"
format_stamp: "Formatted at 2026-08-07 18:07:10 on dist-test-slave-rnrw"
server_key: "d5c31802313edb32e69a1375c58daa91"
server_key_iv: "cf1f8977ee1247b3e2617365cd9618a6"
server_key_version: "encryptionkey@0"
I20260807 18:07:10.755558 18615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:10.761706 18615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:10.762540 18615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:10.762877 18615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:10.786301 18615 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:38229
I20260807 18:07:10.786329 18682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:38229 every 8 connection(s)
I20260807 18:07:10.787993 18615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:07:10.792042 18683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:10.793701 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18615
I20260807 18:07:10.793946 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:07:10.794757 8442 external_mini_cluster.cc:1428] Setting key ffe932281b14f118ccb0395fefa780bb
I20260807 18:07:10.804817 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:10.797323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46322 (local address 127.8.62.190:38229)
0807 18:07:10.798745 (+ 1422us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:10.798758 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:10.798809 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:07:10.800740 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:10.800767 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:10.801245 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:10.801682 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:10.802907 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:10.802917 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:10.802935 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:10.802953 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:10.803017 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:10.803034 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:10.803324 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:10.803471 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 18:07:10.803772 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":816,"thread_start_us":499,"threads_started":1}
I20260807 18:07:10.805809 18683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3: Bootstrap starting.
I20260807 18:07:10.809409 18683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:10.810693 18683 log.cc:824] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:07:10.813859 18683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3: No bootstrap required, opened a new log
I20260807 18:07:10.821957 18683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } }
I20260807 18:07:10.822433 18683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:10.822561 18683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b85523a76434f6f9b19081d7e5176e3, State: Initialized, Role: FOLLOWER
I20260807 18:07:10.823097 18683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } }
I20260807 18:07:10.823319 18683 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:07:10.823421 18683 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:07:10.823560 18683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:10.825986 18683 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } }
I20260807 18:07:10.826723 18683 leader_election.cc:302] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b85523a76434f6f9b19081d7e5176e3; no voters:
I20260807 18:07:10.827576 18683 leader_election.cc:288] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:07:10.827708 18688 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:07:10.828437 18688 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [term 1 LEADER]: Becoming Leader. State: Replica: 3b85523a76434f6f9b19081d7e5176e3, State: Running, Role: LEADER
I20260807 18:07:10.829084 18688 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } }
I20260807 18:07:10.830104 18683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:07:10.832093 18690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b85523a76434f6f9b19081d7e5176e3. Latest consensus state: current_term: 1 leader_uuid: "3b85523a76434f6f9b19081d7e5176e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } } }
I20260807 18:07:10.832674 18690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:10.832391 18689 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b85523a76434f6f9b19081d7e5176e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b85523a76434f6f9b19081d7e5176e3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 38229 } } }
I20260807 18:07:10.833002 18689 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:10.834692 18695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:07:10.840219 18695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:07:10.850713 18695 catalog_manager.cc:1347] Generated new cluster ID: bd58e479841b45f388d3f3993913e9e7
I20260807 18:07:10.850872 18695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:07:10.880992 18695 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:07:10.881541 18695 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:07:10.900576 18695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b85523a76434f6f9b19081d7e5176e3: Generated new TSK 0
I20260807 18:07:10.901669 18695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:07:10.921054 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--builtin_ntp_servers=127.8.62.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 18:07:11.269761 18707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:11.270076 18707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:11.270205 18707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:11.277850 18707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:11.277971 18707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:11.278162 18707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:07:11.287951 18707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:11.289420 18707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:11.291316 18707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:11.309037 18713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:11.308977 18714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:11.309017 18716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:11.310477 18707 server_base.cc:1034] running on GCE node
I20260807 18:07:11.311285 18707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:11.312419 18707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:11.313715 18707 hybrid_clock.cc:648] HybridClock initialized: now 1786126031313627 us; error 71 us; skew 500 ppm
I20260807 18:07:11.314103 18707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:11.316535 18707 webserver.cc:460] Webserver started at http://127.8.62.129:38333/ using document root <none> and password file <none>
I20260807 18:07:11.317344 18707 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:11.317482 18707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:11.317844 18707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:11.320972 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "bf3dfd6e0a3a4caa8f214c0a382a3001"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "f1414e4ccc37e06b7c6611152d7ae192"
server_key_iv: "26b9494972df48424ceafc24209d5ba1"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.321820 18707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "bf3dfd6e0a3a4caa8f214c0a382a3001"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "f1414e4ccc37e06b7c6611152d7ae192"
server_key_iv: "26b9494972df48424ceafc24209d5ba1"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.327514 18707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 18:07:11.331559 18722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.333019 18707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:07:11.333212 18707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "bf3dfd6e0a3a4caa8f214c0a382a3001"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "f1414e4ccc37e06b7c6611152d7ae192"
server_key_iv: "26b9494972df48424ceafc24209d5ba1"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.333449 18707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:11.339488 18707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:11.340334 18707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:11.340688 18707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:11.341619 18707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:11.343101 18707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:11.343233 18707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.343362 18707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:11.343451 18707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.377481 18707 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:39333
I20260807 18:07:11.377537 18837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:39333 every 8 connection(s)
I20260807 18:07:11.378870 18707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:07:11.387781 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18707
I20260807 18:07:11.388010 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:11.381586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:56591 (local address 127.8.62.190:38229)
0807 18:07:11.382111 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:11.382116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:11.383354 (+ 1238us) server_negotiation.cc:407] Connection header received
0807 18:07:11.385254 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:11.385259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:11.385328 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:11.385434 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:11.387020 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:11.387025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:11.387028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:11.387047 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:11.387098 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:11.387111 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:11.387296 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:11.387515 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 18:07:11.387650 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":258,"threads_started":1}
I20260807 18:07:11.388022 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:07:11.388793 8442 external_mini_cluster.cc:1428] Setting key db6b6466e61dca41564c3b3f0750cbb8
I20260807 18:07:11.389143 18841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:11.382221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38229 (local address 127.8.62.129:56591)
0807 18:07:11.383187 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:11.383225 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:11.384939 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:11.385753 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:11.385762 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:11.385871 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:11.386667 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:11.386670 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:11.386716 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:11.386824 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:11.387261 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:11.387270 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:11.387277 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 18:07:11.387599 (+ 322us) client_negotiation.cc:240] Negotiation successful
0807 18:07:11.387842 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":290,"threads_started":1}
I20260807 18:07:11.391083 18838 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38229
I20260807 18:07:11.391415 18838 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:11.392050 18838 heartbeater.cc:511] Master 127.8.62.190:38229 requested a full tablet report, sending...
I20260807 18:07:11.392244 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--builtin_ntp_servers=127.8.62.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:07:11.394488 18647 ts_manager.cc:194] Registered new tserver with Master: bf3dfd6e0a3a4caa8f214c0a382a3001 (127.8.62.129:39333)
W20260807 18:07:11.741426 18842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:11.741736 18842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:11.741904 18842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:11.749331 18842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:11.749454 18842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:11.749656 18842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:07:11.759012 18842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:11.760473 18842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:11.762374 18842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:11.776129 18847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:11.776134 18848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:11.776018 18850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:11.777359 18842 server_base.cc:1034] running on GCE node
I20260807 18:07:11.778008 18842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:11.778980 18842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:11.780189 18842 hybrid_clock.cc:648] HybridClock initialized: now 1786126031780127 us; error 46 us; skew 500 ppm
I20260807 18:07:11.780519 18842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:11.783004 18842 webserver.cc:460] Webserver started at http://127.8.62.130:37519/ using document root <none> and password file <none>
I20260807 18:07:11.783733 18842 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:11.783828 18842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:11.784204 18842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:11.787177 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "007d0cc9c3884d9d813b0d16136dae31"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "5e1c379ecb4da4114a4a732f996b35c1"
server_key_iv: "6c274a55ab27e0aede150a3d43315f58"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.788008 18842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "007d0cc9c3884d9d813b0d16136dae31"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "5e1c379ecb4da4114a4a732f996b35c1"
server_key_iv: "6c274a55ab27e0aede150a3d43315f58"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.793465 18842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 18:07:11.797158 18856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.798566 18842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:07:11.798751 18842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "007d0cc9c3884d9d813b0d16136dae31"
format_stamp: "Formatted at 2026-08-07 18:07:11 on dist-test-slave-rnrw"
server_key: "5e1c379ecb4da4114a4a732f996b35c1"
server_key_iv: "6c274a55ab27e0aede150a3d43315f58"
server_key_version: "encryptionkey@0"
I20260807 18:07:11.798974 18842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:11.805071 18842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:11.805878 18842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:11.806160 18842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:11.807096 18842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:11.808627 18842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:11.808723 18842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.808871 18842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:11.808965 18842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:11.844014 18842 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:38401
I20260807 18:07:11.844108 18971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:38401 every 8 connection(s)
I20260807 18:07:11.845482 18842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:07:11.848680 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18842
I20260807 18:07:11.848903 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:07:11.849380 8442 external_mini_cluster.cc:1428] Setting key 74361db4e1678e3b60605905b3411feb
I20260807 18:07:11.852769 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--builtin_ntp_servers=127.8.62.148:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 18:07:11.860905 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:11.848247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:56293 (local address 127.8.62.190:38229)
0807 18:07:11.848494 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:11.848503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:11.850296 (+ 1793us) server_negotiation.cc:407] Connection header received
0807 18:07:11.852362 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:11.852371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:11.852446 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:11.852606 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:11.859769 (+ 7163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:11.859778 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:11.859787 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:11.859820 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:11.859873 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:11.859895 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:11.860020 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:11.860327 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 18:07:11.860445 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:07:11.862275 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:11.848907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38229 (local address 127.8.62.130:56293)
0807 18:07:11.850120 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:11.850169 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:11.852071 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:11.852863 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:11.852872 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:11.853069 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:11.859321 (+ 6252us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:11.859328 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:11.859400 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:11.859536 (+ 136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:11.860024 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:11.860042 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:11.860052 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 18:07:11.860448 (+ 396us) client_negotiation.cc:240] Negotiation successful
0807 18:07:11.860855 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":371,"threads_started":1}
I20260807 18:07:11.864004 18972 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38229
I20260807 18:07:11.864398 18972 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:11.865172 18972 heartbeater.cc:511] Master 127.8.62.190:38229 requested a full tablet report, sending...
I20260807 18:07:11.866888 18647 ts_manager.cc:194] Registered new tserver with Master: 007d0cc9c3884d9d813b0d16136dae31 (127.8.62.130:38401)
W20260807 18:07:12.213316 18975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:12.213662 18975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:12.213828 18975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:12.221407 18975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:12.221553 18975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:12.221741 18975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:07:12.232074 18975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:38229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:12.233542 18975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:12.235357 18975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:12.249784 18980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:12.249866 18981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:12.249879 18983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:12.251183 18975 server_base.cc:1034] running on GCE node
I20260807 18:07:12.251858 18975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:12.252890 18975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:12.254159 18975 hybrid_clock.cc:648] HybridClock initialized: now 1786126032254099 us; error 73 us; skew 500 ppm
I20260807 18:07:12.254518 18975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:07:12.256975 18975 webserver.cc:460] Webserver started at http://127.8.62.131:41193/ using document root <none> and password file <none>
I20260807 18:07:12.257706 18975 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:12.257844 18975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:12.258194 18975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:12.261433 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "603e5bbd8a2c43aab22a69d3bdc5e7b5"
format_stamp: "Formatted at 2026-08-07 18:07:12 on dist-test-slave-rnrw"
server_key: "e768dd04f405ea790e470c8d069ee5b5"
server_key_iv: "c2ce785f7dd4d3ad4a25806639dde0e6"
server_key_version: "encryptionkey@0"
I20260807 18:07:12.262364 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "603e5bbd8a2c43aab22a69d3bdc5e7b5"
format_stamp: "Formatted at 2026-08-07 18:07:12 on dist-test-slave-rnrw"
server_key: "e768dd04f405ea790e470c8d069ee5b5"
server_key_iv: "c2ce785f7dd4d3ad4a25806639dde0e6"
server_key_version: "encryptionkey@0"
I20260807 18:07:12.268074 18975 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:07:12.272500 18989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:12.274083 18975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:07:12.274308 18975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "603e5bbd8a2c43aab22a69d3bdc5e7b5"
format_stamp: "Formatted at 2026-08-07 18:07:12 on dist-test-slave-rnrw"
server_key: "e768dd04f405ea790e470c8d069ee5b5"
server_key_iv: "c2ce785f7dd4d3ad4a25806639dde0e6"
server_key_version: "encryptionkey@0"
I20260807 18:07:12.274605 18975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:12.280977 18975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:12.281970 18975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:12.282300 18975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:12.283366 18975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:12.284983 18975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:12.285128 18975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:12.285274 18975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:12.285367 18975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:12.322271 18975 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:36771
I20260807 18:07:12.322331 19104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:36771 every 8 connection(s)
I20260807 18:07:12.323726 18975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:07:12.330844 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 18975
I20260807 18:07:12.331089 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:07:12.331645 8442 external_mini_cluster.cc:1428] Setting key cd42f72ede2fc053246d26a72cb4cf9f
I20260807 18:07:12.333348 18840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:12.326805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:37799 (local address 127.8.62.190:38229)
0807 18:07:12.326996 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:12.327001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:12.328679 (+ 1678us) server_negotiation.cc:407] Connection header received
0807 18:07:12.330503 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:12.330509 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:12.330600 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:12.330717 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:12.332307 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:12.332312 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:12.332316 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:12.332343 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:12.332408 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:12.332505 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:12.332661 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:12.332947 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 18:07:12.333017 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:07:12.334607 19107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:12.327465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:38229 (local address 127.8.62.131:37799)
0807 18:07:12.328449 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:12.328488 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:12.330192 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:12.331006 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:12.331021 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:12.331173 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:12.331941 (+ 768us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:07:12.331944 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:12.331984 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:07:12.332090 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:12.332657 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:12.332666 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:12.332672 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:07:12.332997 (+ 325us) client_negotiation.cc:240] Negotiation successful
0807 18:07:12.333299 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260807 18:07:12.336127 19105 heartbeater.cc:347] Connected to a master server at 127.8.62.190:38229
I20260807 18:07:12.336475 19105 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:12.337121 19105 heartbeater.cc:511] Master 127.8.62.190:38229 requested a full tablet report, sending...
I20260807 18:07:12.338650 18647 ts_manager.cc:194] Registered new tserver with Master: 603e5bbd8a2c43aab22a69d3bdc5e7b5 (127.8.62.131:36771)
I20260807 18:07:12.348014 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 18:07:12.355146 18840 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:07:12.353261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46330 (local address 127.8.62.190:38229)
0807 18:07:12.353520 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:12.353525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:12.354031 (+ 506us) server_negotiation.cc:407] Connection header received
0807 18:07:12.354227 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:12.354231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:12.354289 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:12.354386 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:12.354854 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46330: BlockingRecv error: recv got EOF from 127.0.0.1:46330 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:07:12.356068 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18707
I20260807 18:07:12.359764 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:12.357117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.8.62.129:39333)
0807 18:07:12.357650 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:12.357657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:12.357684 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:07:12.357748 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:12.357755 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:12.357969 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:12.358106 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:12.359082 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:12.359089 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:12.359101 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:12.359115 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:12.359158 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:12.359173 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:12.359288 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:12.359357 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 18:07:12.359438 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":239,"threads_started":1}
I20260807 18:07:12.361195 18832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:12.443998 18838 heartbeater.cc:503] Master 127.8.62.190:38229 was elected leader, sending a full tablet report...
I20260807 18:07:12.445072 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18707
W20260807 18:07:12.464115 18632 connection.cc:535] server connection from 127.8.62.129:56591 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:07:12.464273 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18842
I20260807 18:07:12.468310 19116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:12.465425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.8.62.130:38401)
0807 18:07:12.466112 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:12.466119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:12.466148 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:07:12.466209 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:12.466215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:12.466435 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:12.466587 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:12.467549 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:12.467557 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:12.467574 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:12.467593 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:12.467644 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:12.467661 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:12.467776 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:12.467857 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:07:12.467976 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":362,"threads_started":1}
I20260807 18:07:12.469626 18967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:12.553633 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18842
I20260807 18:07:12.572705 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18975
I20260807 18:07:12.576906 19118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:12.573803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.8.62.131:36771)
0807 18:07:12.574471 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:12.574478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:12.574511 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:07:12.574574 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:12.574580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:12.574795 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:12.574922 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:12.575977 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:12.575983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:12.576015 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:07:12.576032 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:12.576072 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:07:12.576087 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:12.576223 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:12.576279 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 18:07:12.576390 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":357,"threads_started":1}
I20260807 18:07:12.578169 19100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:12.660681 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18975
I20260807 18:07:12.680198 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 18615
I20260807 18:07:12.681639 18678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:12.788666 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 18615
2026-08-07T18:07:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2515 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:07:12 dist-test-slave-rnrw krb5kdc[19123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:07:12 dist-test-slave-rnrw krb5kdc[19123](info): set up 2 sockets
Aug 07 18:07:12 dist-test-slave-rnrw krb5kdc[19123](info): commencing operation
krb5kdc: starting...
W20260807 18:07:13.984741 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.016s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:07:14 dist-test-slave-rnrw krb5kdc[19123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:07:14.244868 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:45423
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:45423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:07:14.593096 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:14.593392 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:14.593580 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:14.600782 19136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:07:14.600889 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:14.601016 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:14.601114 19136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:07:14.601197 19136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:07:14.611035 19136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:45423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:45423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:14.612399 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:14.613771 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:14.625677 19144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:14.625742 19142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:14.625780 19141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:14.626756 19136 server_base.cc:1034] running on GCE node
I20260807 18:07:14.627532 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:14.629027 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:14.630266 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786126034630213 us; error 45 us; skew 500 ppm
I20260807 18:07:14.630635 19136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:07:14 dist-test-slave-rnrw krb5kdc[19123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:07:14.639456 19136 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:07:14.641255 19136 webserver.cc:460] Webserver started at http://127.8.62.190:44015/ using document root <none> and password file <none>
I20260807 18:07:14.642009 19136 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:14.642155 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:14.642483 19136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:14.645673 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "5af68823ab43411085fc442ccef4cf97"
format_stamp: "Formatted at 2026-08-07 18:07:14 on dist-test-slave-rnrw"
server_key: "d540b0547425471c2dde6115f42ca517"
server_key_iv: "8b9a9d7f88c28b1a3466670dcb26ebb9"
server_key_version: "encryptionkey@0"
I20260807 18:07:14.646469 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "5af68823ab43411085fc442ccef4cf97"
format_stamp: "Formatted at 2026-08-07 18:07:14 on dist-test-slave-rnrw"
server_key: "d540b0547425471c2dde6115f42ca517"
server_key_iv: "8b9a9d7f88c28b1a3466670dcb26ebb9"
server_key_version: "encryptionkey@0"
I20260807 18:07:14.651983 19136 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:07:14.655575 19151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:14.657119 19136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:07:14.657310 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "5af68823ab43411085fc442ccef4cf97"
format_stamp: "Formatted at 2026-08-07 18:07:14 on dist-test-slave-rnrw"
server_key: "d540b0547425471c2dde6115f42ca517"
server_key_iv: "8b9a9d7f88c28b1a3466670dcb26ebb9"
server_key_version: "encryptionkey@0"
I20260807 18:07:14.657521 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:14.678885 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:14.683995 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:14.684322 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:14.704770 19136 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:45423
I20260807 18:07:14.704818 19205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:45423 every 8 connection(s)
I20260807 18:07:14.706316 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:07:14.710505 19206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:14.712457 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 19136
I20260807 18:07:14.712736 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:07:14.713601 8442 external_mini_cluster.cc:1428] Setting key ff6a9a7e5e0f6d3607f44b3fde068f3d
I20260807 18:07:14.723790 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: Bootstrap starting.
I20260807 18:07:14.727265 19206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:14.728381 19206 log.cc:824] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:07:14 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:07:14.731999 19206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: No bootstrap required, opened a new log
I20260807 18:07:14.740190 19209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:14.716105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.8.62.190:45423)
0807 18:07:14.717146 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:14.717162 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:14.717219 (+ 57us) server_negotiation.cc:407] Connection header received
0807 18:07:14.719246 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:14.719274 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:14.719787 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:14.720274 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:14.722105 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:14.723595 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:14.725199 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:14.726013 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:14.726250 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:14.731619 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:14.731628 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:14.731644 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:14.731701 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:14.735293 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:14.736217 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:14.736225 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:14.736232 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:14.736436 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:14.736926 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:14.736930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:14.736932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:14.737575 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:14.737951 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:14.738509 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 18:07:14.739019 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":775,"thread_start_us":490,"threads_started":1}
I20260807 18:07:14.740890 19206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } }
I20260807 18:07:14.741324 19206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:14.741449 19206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af68823ab43411085fc442ccef4cf97, State: Initialized, Role: FOLLOWER
I20260807 18:07:14.742166 19206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } }
I20260807 18:07:14.742429 19206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:07:14.742597 19206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:07:14.742789 19206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:14.745427 19206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } }
I20260807 18:07:14.746048 19206 leader_election.cc:302] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af68823ab43411085fc442ccef4cf97; no voters:
I20260807 18:07:14.746678 19206 leader_election.cc:288] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:07:14.746840 19211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:07:14.748476 19211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [term 1 LEADER]: Becoming Leader. State: Replica: 5af68823ab43411085fc442ccef4cf97, State: Running, Role: LEADER
I20260807 18:07:14.749171 19206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:07:14.749116 19211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } }
I20260807 18:07:14.752148 19213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af68823ab43411085fc442ccef4cf97. Latest consensus state: current_term: 1 leader_uuid: "5af68823ab43411085fc442ccef4cf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } } }
I20260807 18:07:14.752532 19213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:14.752269 19212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af68823ab43411085fc442ccef4cf97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af68823ab43411085fc442ccef4cf97" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45423 } } }
I20260807 18:07:14.752758 19212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97 [sys.catalog]: This master's current role is: LEADER
I20260807 18:07:14.759938 19218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:07:14.765834 19218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:07:14.776497 19218 catalog_manager.cc:1347] Generated new cluster ID: 8efc706887bc44909965c70cf6ab0bb6
I20260807 18:07:14.776675 19218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:07:14.793107 19218 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:07:14.794524 19218 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:07:14.804623 19218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: Generated new TSK 0
I20260807 18:07:14.805581 19218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:07:14.872746 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:07:15.224301 19232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:15.224632 19232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:15.224740 19232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:15.232928 19232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:15.233049 19232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:15.233166 19232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:07:15.243507 19232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:15.244871 19232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:15.246160 19232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:15.260571 19238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:15.260609 19240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:15.260668 19237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:15.266716 19232 server_base.cc:1034] running on GCE node
I20260807 18:07:15.267577 19232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:15.268899 19232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:15.270215 19232 hybrid_clock.cc:648] HybridClock initialized: now 1786126035270143 us; error 80 us; skew 500 ppm
I20260807 18:07:15.270612 19232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:07:15 dist-test-slave-rnrw krb5kdc[19123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126035, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:07:15.274371 19232 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:07:15.276098 19232 webserver.cc:460] Webserver started at http://127.8.62.129:37101/ using document root <none> and password file <none>
I20260807 18:07:15.276961 19232 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:15.277076 19232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:15.277343 19232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:15.280717 19232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "1b81ddc056e234eb6bf5c2863b0a78a2"
server_key_iv: "31010dbe19779437893be02b1bc553ea"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.281601 19232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "1b81ddc056e234eb6bf5c2863b0a78a2"
server_key_iv: "31010dbe19779437893be02b1bc553ea"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.287662 19232 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:07:15.291870 19248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:15.293684 19232 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:07:15.293893 19232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "1b81ddc056e234eb6bf5c2863b0a78a2"
server_key_iv: "31010dbe19779437893be02b1bc553ea"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.294106 19232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:15.308830 19232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:15.313624 19232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:15.313956 19232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:15.314976 19232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:15.316392 19232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:15.316468 19232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:15.316641 19232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:15.316715 19232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:07:15.352077 19232 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:40039
I20260807 18:07:15.352273 19362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:40039 every 8 connection(s)
I20260807 18:07:15.353516 19232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:07:15.360754 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 19232
I20260807 18:07:15.361027 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:07:15.361850 8442 external_mini_cluster.cc:1428] Setting key 31abf7ea7cc81ec141dfe8ac11205288
Aug 07 18:07:15 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126035, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:07:15.377306 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:15.356236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:55491 (local address 127.8.62.190:45423)
0807 18:07:15.356751 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:15.356757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:15.357752 (+ 995us) server_negotiation.cc:407] Connection header received
0807 18:07:15.359566 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:15.359571 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:15.359684 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:15.359788 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:15.361992 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.362941 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.364963 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.365732 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.365892 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:15.369825 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:15.369835 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:15.369843 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:15.369896 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:15.373342 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:15.374521 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:15.374525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:15.374527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:15.374651 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:15.375234 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:15.375239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:15.375241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:15.375614 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:15.375753 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:15.376829 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0807 18:07:15.377031 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":230,"threads_started":1}
I20260807 18:07:15.379068 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:15.356829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45423 (local address 127.8.62.129:55491)
0807 18:07:15.357576 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:15.357620 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:15.359299 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:15.360016 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:15.360025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:15.360871 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:15.361770 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:15.361799 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.363129 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.363134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:15.364802 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:15.364816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.365969 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.365975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:15.366176 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:15.366907 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:15.366931 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:15.369528 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:15.373602 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:15.373618 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:15.373641 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:15.374373 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:15.374760 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:15.374768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:15.374774 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:15.375057 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:15.375763 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:15.375775 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:15.376280 (+ 505us) client_negotiation.cc:769] Sending connection context
0807 18:07:15.376720 (+ 440us) client_negotiation.cc:240] Negotiation successful
0807 18:07:15.377417 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":279,"threads_started":1}
I20260807 18:07:15.380884 19363 heartbeater.cc:347] Connected to a master server at 127.8.62.190:45423
I20260807 18:07:15.381307 19363 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:15.382299 19363 heartbeater.cc:511] Master 127.8.62.190:45423 requested a full tablet report, sending...
I20260807 18:07:15.385063 19169 ts_manager.cc:194] Registered new tserver with Master: b9ee55440e0046f6b8a2eb8aa0f040f7 (127.8.62.129:40039)
I20260807 18:07:15.386771 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:55491
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:07:15.424808 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:07:15.786706 19369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:15.787081 19369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:15.787267 19369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:15.794826 19369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:15.795017 19369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:15.795222 19369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:07:15.805521 19369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:15.806914 19369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:15.808324 19369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:15.821815 19376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:15.821961 19378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:15.822739 19380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:15.822799 19369 server_base.cc:1034] running on GCE node
I20260807 18:07:15.823482 19369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:15.824469 19369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:15.825754 19369 hybrid_clock.cc:648] HybridClock initialized: now 1786126035825713 us; error 32 us; skew 500 ppm
I20260807 18:07:15.826112 19369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:07:15 dist-test-slave-rnrw krb5kdc[19123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126035, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:07:15.829579 19369 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:07:15.831074 19369 webserver.cc:460] Webserver started at http://127.8.62.130:33023/ using document root <none> and password file <none>
I20260807 18:07:15.831800 19369 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:15.831956 19369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:15.832283 19369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:15.835391 19369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "3c52bfbcef564636ad56cbf737af90e0"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "267041de3782dbb77641aa5898393c4d"
server_key_iv: "d12d9bb1ace11c6647f77825563d7111"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.836227 19369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "3c52bfbcef564636ad56cbf737af90e0"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "267041de3782dbb77641aa5898393c4d"
server_key_iv: "d12d9bb1ace11c6647f77825563d7111"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.841878 19369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:07:15.846160 19387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:15.847932 19369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:07:15.848177 19369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "3c52bfbcef564636ad56cbf737af90e0"
format_stamp: "Formatted at 2026-08-07 18:07:15 on dist-test-slave-rnrw"
server_key: "267041de3782dbb77641aa5898393c4d"
server_key_iv: "d12d9bb1ace11c6647f77825563d7111"
server_key_version: "encryptionkey@0"
I20260807 18:07:15.848425 19369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:15.867846 19369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:15.872509 19369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:15.872920 19369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:15.873986 19369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:15.875466 19369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:15.875577 19369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:15.875744 19369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:15.875819 19369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:15.912715 19369 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:38049
I20260807 18:07:15.912853 19501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:38049 every 8 connection(s)
I20260807 18:07:15.914131 19369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:07:15.924124 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 19369
I20260807 18:07:15.924347 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:07:15.924921 8442 external_mini_cluster.cc:1428] Setting key 0c5a6bf41da8f19d5c6b8072b2131667
Aug 07 18:07:15 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126035, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:07:15.936821 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:15.917160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:57891 (local address 127.8.62.190:45423)
0807 18:07:15.917636 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:15.917641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:15.918732 (+ 1091us) server_negotiation.cc:407] Connection header received
0807 18:07:15.920673 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:15.920678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:15.920753 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:15.920852 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:15.922467 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.923239 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.925637 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.926173 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.926303 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:15.930531 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:15.930536 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:15.930540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:15.930573 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:15.933399 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:15.934218 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:15.934227 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:15.934232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:15.934377 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:15.934849 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:15.934856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:15.934859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:15.935260 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:15.935431 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:15.936217 (+ 786us) server_negotiation.cc:299] Negotiation successful
0807 18:07:15.936464 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":238,"threads_started":1}
I20260807 18:07:15.937767 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:15.917753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45423 (local address 127.8.62.130:57891)
0807 18:07:15.918558 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:15.918606 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:15.920299 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:15.921074 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:15.921082 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:15.921686 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:15.922287 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:15.922306 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.923386 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.923390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:15.925462 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:15.925477 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:15.926388 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:15.926393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:15.926598 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:15.927402 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:15.927454 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:15.930247 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:15.933563 (+ 3316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:15.933573 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:15.933592 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:15.934081 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:15.934499 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:15.934504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:15.934508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:15.934721 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:15.935412 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:15.935418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:15.935820 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 18:07:15.936077 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 18:07:15.936510 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":297,"threads_started":1}
I20260807 18:07:15.939445 19502 heartbeater.cc:347] Connected to a master server at 127.8.62.190:45423
I20260807 18:07:15.939814 19502 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:15.940685 19502 heartbeater.cc:511] Master 127.8.62.190:45423 requested a full tablet report, sending...
I20260807 18:07:15.942266 19169 ts_manager.cc:194] Registered new tserver with Master: 3c52bfbcef564636ad56cbf737af90e0 (127.8.62.130:38049)
I20260807 18:07:15.943130 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:57891
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:07:15.985607 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:33153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:07:16.340965 19508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:07:16.341269 19508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:07:16.341430 19508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:07:16.348963 19508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:07:16.349092 19508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:07:16.349308 19508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:07:16.362499 19508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:45423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:07:16.363803 19508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:07:16.365227 19508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:07:16.378710 19513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:16.378760 19514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:07:16.378794 19516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:07:16.379629 19508 server_base.cc:1034] running on GCE node
I20260807 18:07:16.380415 19508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:07:16.381458 19508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:07:16.382725 19508 hybrid_clock.cc:648] HybridClock initialized: now 1786126036382661 us; error 64 us; skew 500 ppm
I20260807 18:07:16.383054 19508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:07:16 dist-test-slave-rnrw krb5kdc[19123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126036, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:07:16.386436 19508 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:07:16.388025 19508 webserver.cc:460] Webserver started at http://127.8.62.131:40621/ using document root <none> and password file <none>
I20260807 18:07:16.388810 19508 fs_manager.cc:359] Metadata directory not provided
I20260807 18:07:16.388958 19508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:07:16.389267 19508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:07:16.390846 19363 heartbeater.cc:503] Master 127.8.62.190:45423 was elected leader, sending a full tablet report...
I20260807 18:07:16.392791 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "07eca32d132b45e3803b9b9008a2361b"
format_stamp: "Formatted at 2026-08-07 18:07:16 on dist-test-slave-rnrw"
server_key: "53358e68b15bdbcbaa21b32bd6dbc55d"
server_key_iv: "f67c6039edd235051293528f6210725c"
server_key_version: "encryptionkey@0"
I20260807 18:07:16.393620 19508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "07eca32d132b45e3803b9b9008a2361b"
format_stamp: "Formatted at 2026-08-07 18:07:16 on dist-test-slave-rnrw"
server_key: "53358e68b15bdbcbaa21b32bd6dbc55d"
server_key_iv: "f67c6039edd235051293528f6210725c"
server_key_version: "encryptionkey@0"
I20260807 18:07:16.399047 19508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:07:16.402626 19523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:16.404045 19508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:07:16.404232 19508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "07eca32d132b45e3803b9b9008a2361b"
format_stamp: "Formatted at 2026-08-07 18:07:16 on dist-test-slave-rnrw"
server_key: "53358e68b15bdbcbaa21b32bd6dbc55d"
server_key_iv: "f67c6039edd235051293528f6210725c"
server_key_version: "encryptionkey@0"
I20260807 18:07:16.404448 19508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:07:16.427186 19508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:07:16.432296 19508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:07:16.432720 19508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:07:16.433743 19508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:07:16.435182 19508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:07:16.435292 19508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:16.435436 19508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:07:16.435516 19508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:07:16.471036 19508 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:40221
I20260807 18:07:16.471130 19637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:40221 every 8 connection(s)
I20260807 18:07:16.472469 19508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:07:16.482681 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 19508
I20260807 18:07:16.482933 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:07:16.483464 8442 external_mini_cluster.cc:1428] Setting key 791fa4429b71f1e1800b9901fcf1ef77
Aug 07 18:07:16 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126036, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:07:16.494235 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.475155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:37633 (local address 127.8.62.190:45423)
0807 18:07:16.475645 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.475653 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.476758 (+ 1105us) server_negotiation.cc:407] Connection header received
0807 18:07:16.479246 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.479252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.479372 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.479507 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:16.481197 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.481989 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.483799 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.484329 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.484488 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.488159 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:16.488165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:16.488168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:16.488204 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:16.490967 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.491763 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.491768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.491770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.491894 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.492349 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.492354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.492356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.492709 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:16.492871 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.493599 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.493829 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":210,"threads_started":1}
I20260807 18:07:16.495251 19641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.475686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45423 (local address 127.8.62.131:37633)
0807 18:07:16.476512 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.476603 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.478930 (+ 2327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.479770 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.479779 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.480320 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:07:16.481004 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.481027 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.482152 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.482157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.483609 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.483645 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.484614 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.484625 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.484878 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.485625 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:07:16.485649 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:07:16.487926 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:07:16.491129 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:16.491139 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:16.491165 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:16.491619 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:16.492020 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:07:16.492025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:07:16.492035 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:07:16.492219 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:07:16.492832 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:07:16.492841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:07:16.493213 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.493497 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.494067 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":302,"threads_started":1}
I20260807 18:07:16.497265 19638 heartbeater.cc:347] Connected to a master server at 127.8.62.190:45423
I20260807 18:07:16.497643 19638 heartbeater.cc:464] Registering TS with master...
I20260807 18:07:16.498648 19638 heartbeater.cc:511] Master 127.8.62.190:45423 requested a full tablet report, sending...
I20260807 18:07:16.500633 19169 ts_manager.cc:194] Registered new tserver with Master: 07eca32d132b45e3803b9b9008a2361b (127.8.62.131:40221)
I20260807 18:07:16.501946 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:37633
I20260807 18:07:16.512774 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:07:16.530673 19640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.517488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.8.62.190:45423)
0807 18:07:16.517702 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.517707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.518078 (+ 371us) server_negotiation.cc:407] Connection header received
0807 18:07:16.518270 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.518274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.518361 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.518452 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:16.519877 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.520707 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.521957 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.522419 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.522555 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.524921 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:16.524925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:16.524929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:16.524971 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:16.527788 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.528680 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.528685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.528687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.528826 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.529251 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.529254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.529256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.529629 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:16.529835 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.530234 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.530421 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260807 18:07:16.535631 19169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:07:16.542856 19169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:07:16.578892 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.556620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.8.62.129:40039)
0807 18:07:16.570449 (+ 13829us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.570456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.570491 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:07:16.570580 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.570586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.570806 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.571051 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:07:16.572309 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.573564 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.576496 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.577611 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.577746 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.578002 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.578339 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.578534 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13669,"thread_start_us":13541,"threads_started":1}
I20260807 18:07:16.578989 19652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.556322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:40039 (local address 127.0.0.1:57026)
0807 18:07:16.557211 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.557262 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.557466 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.571081 (+ 13615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.571089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.571134 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:07:16.571643 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.571656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.573701 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.573709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.576340 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.576350 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.577821 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.577829 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.578044 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.578135 (+ 91us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.578295 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.578482 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":104,"thread_start_us":413,"threads_started":1}
I20260807 18:07:16.580433 19656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.557658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:38049 (local address 127.0.0.1:40158)
0807 18:07:16.571146 (+ 13488us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.571171 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.571355 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.572732 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.572754 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.572783 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:07:16.573181 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.573199 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.575112 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.575117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.577321 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.577336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.579587 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.579594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.579777 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.579824 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.579900 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.580053 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13304,"mutex_wait_us":25,"thread_start_us":13625,"threads_started":1}
I20260807 18:07:16.581193 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.557964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40158 (local address 127.8.62.130:38049)
0807 18:07:16.570614 (+ 12650us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.570627 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.571746 (+ 1119us) server_negotiation.cc:407] Connection header received
0807 18:07:16.571893 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.571904 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.572198 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.572410 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:07:16.573533 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.574945 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.577495 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.579004 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.579149 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.579339 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.580762 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.580950 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12417,"thread_start_us":13487,"threads_started":1}
I20260807 18:07:16.584051 19298 tablet_service.cc:1467] Processing CreateTablet for tablet 793448e1de3b46cdaffe01d508e84da1 (DEFAULT_TABLE table=test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:07:16.585124 19654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.557063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:40221 (local address 127.0.0.1:60048)
0807 18:07:16.569500 (+ 12437us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.569533 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.569751 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.575087 (+ 5336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.575092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.575471 (+ 379us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:07:16.576035 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.576047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.577951 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.577955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.580657 (+ 2702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.580672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.584257 (+ 3585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.584263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.584435 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.584466 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.584605 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.584739 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12217,"mutex_wait_us":1,"thread_start_us":12824,"threads_started":1}
I20260807 18:07:16.585180 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.557273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.8.62.131:40221)
0807 18:07:16.574226 (+ 16953us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.574235 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.574269 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:07:16.574367 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.574374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.574618 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.574811 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:07:16.576246 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.577742 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.582364 (+ 4622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.583776 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.583902 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.584116 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.584666 (+ 550us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.584832 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16785,"thread_start_us":16624,"threads_started":1}
I20260807 18:07:16.584854 19436 tablet_service.cc:1467] Processing CreateTablet for tablet 793448e1de3b46cdaffe01d508e84da1 (DEFAULT_TABLE table=test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:07:16.585731 19298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793448e1de3b46cdaffe01d508e84da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:16.585984 19436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793448e1de3b46cdaffe01d508e84da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:16.588768 19573 tablet_service.cc:1467] Processing CreateTablet for tablet 793448e1de3b46cdaffe01d508e84da1 (DEFAULT_TABLE table=test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:07:16.590128 19573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793448e1de3b46cdaffe01d508e84da1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:07:16.597551 19658 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Bootstrap starting.
I20260807 18:07:16.601011 19659 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Bootstrap starting.
I20260807 18:07:16.601269 19658 tablet_bootstrap.cc:654] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:16.602931 19658 log.cc:824] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Log is configured to *not* fsync() on all Append() calls
I20260807 18:07:16.604799 19659 tablet_bootstrap.cc:654] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:16.606065 19659 log.cc:824] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Log is configured to *not* fsync() on all Append() calls
I20260807 18:07:16.606377 19658 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: No bootstrap required, opened a new log
I20260807 18:07:16.606709 19658 ts_tablet_manager.cc:1397] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:07:16.609216 19659 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: No bootstrap required, opened a new log
I20260807 18:07:16.609493 19659 ts_tablet_manager.cc:1397] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260807 18:07:16.612666 19661 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Bootstrap starting.
I20260807 18:07:16.615796 19661 tablet_bootstrap.cc:654] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Neither blocks nor log segments found. Creating new log.
I20260807 18:07:16.616911 19661 log.cc:824] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:07:16.617529 19658 raft_consensus.cc:348] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.618063 19658 raft_consensus.cc:374] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:16.618167 19658 raft_consensus.cc:729] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9ee55440e0046f6b8a2eb8aa0f040f7, State: Initialized, Role: FOLLOWER
I20260807 18:07:16.618911 19658 consensus_queue.cc:260] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.620870 19658 ts_tablet_manager.cc:1428] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260807 18:07:16.621496 19659 raft_consensus.cc:348] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.621989 19659 raft_consensus.cc:374] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:16.622095 19659 raft_consensus.cc:729] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c52bfbcef564636ad56cbf737af90e0, State: Initialized, Role: FOLLOWER
I20260807 18:07:16.622836 19659 consensus_queue.cc:260] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.624169 19502 heartbeater.cc:503] Master 127.8.62.190:45423 was elected leader, sending a full tablet report...
I20260807 18:07:16.624855 19659 ts_tablet_manager.cc:1428] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260807 18:07:16.627846 19661 tablet_bootstrap.cc:492] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: No bootstrap required, opened a new log
I20260807 18:07:16.628196 19661 ts_tablet_manager.cc:1397] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.008s
I20260807 18:07:16.635922 19661 raft_consensus.cc:348] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.636320 19661 raft_consensus.cc:374] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:07:16.636395 19661 raft_consensus.cc:729] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07eca32d132b45e3803b9b9008a2361b, State: Initialized, Role: FOLLOWER
I20260807 18:07:16.637070 19661 consensus_queue.cc:260] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.638186 19638 heartbeater.cc:503] Master 127.8.62.190:45423 was elected leader, sending a full tablet report...
I20260807 18:07:16.638677 19661 ts_tablet_manager.cc:1428] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Time spent starting tablet: real 0.010s user 0.008s sys 0.004s
W20260807 18:07:16.666303 19503 tablet.cc:2369] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:07:16.724188 19639 tablet.cc:2369] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:07:16.750590 19664 raft_consensus.cc:482] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:07:16.751048 19664 raft_consensus.cc:504] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.753082 19664 leader_election.cc:288] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c52bfbcef564636ad56cbf737af90e0 (127.8.62.130:38049), 07eca32d132b45e3803b9b9008a2361b (127.8.62.131:40221)
I20260807 18:07:16.762899 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.753588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:36485 (local address 127.8.62.130:38049)
0807 18:07:16.753843 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.753853 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.754353 (+ 500us) server_negotiation.cc:407] Connection header received
0807 18:07:16.754607 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.754613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.754695 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.754802 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:07:16.755636 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.756805 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.760220 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.761566 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.761768 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.761849 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.762462 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.762613 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 18:07:16.762899 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.753659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:35377 (local address 127.8.62.131:40221)
0807 18:07:16.753881 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.753889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.754541 (+ 652us) server_negotiation.cc:407] Connection header received
0807 18:07:16.754621 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.754626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.754695 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.754803 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:07:16.755636 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.756815 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.760220 (+ 3405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.761644 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.761790 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.761880 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.762469 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.762655 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:07:16.763679 19668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.753451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:40221 (local address 127.8.62.129:35377)
0807 18:07:16.754366 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.754382 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.754494 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.754839 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.754844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.754871 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:07:16.755444 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.755453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.757077 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.757082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.760009 (+ 2927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.760026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.761866 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.761872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.762078 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.762291 (+ 213us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.763125 (+ 834us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.763274 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":457,"thread_start_us":622,"threads_started":1}
I20260807 18:07:16.763872 19667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.753353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:38049 (local address 127.8.62.129:36485)
0807 18:07:16.754188 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 18:07:16.754219 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:07:16.754400 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:07:16.754839 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:07:16.754844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:07:16.754940 (+ 96us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:07:16.755437 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.755450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.757078 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.757082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.760008 (+ 2926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:07:16.760026 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.761854 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.761862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:07:16.762165 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.762273 (+ 108us) client_negotiation.cc:769] Sending connection context
0807 18:07:16.763352 (+ 1079us) client_negotiation.cc:240] Negotiation successful
0807 18:07:16.763500 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":181,"spinlock_wait_cycles":22400,"thread_start_us":494,"threads_started":1}
I20260807 18:07:16.764881 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793448e1de3b46cdaffe01d508e84da1" candidate_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c52bfbcef564636ad56cbf737af90e0" is_pre_election: true
I20260807 18:07:16.764866 19593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793448e1de3b46cdaffe01d508e84da1" candidate_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07eca32d132b45e3803b9b9008a2361b" is_pre_election: true
I20260807 18:07:16.765416 19456 raft_consensus.cc:2454] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ee55440e0046f6b8a2eb8aa0f040f7 in term 0.
I20260807 18:07:16.765417 19593 raft_consensus.cc:2454] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9ee55440e0046f6b8a2eb8aa0f040f7 in term 0.
I20260807 18:07:16.766117 19250 leader_election.cc:302] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07eca32d132b45e3803b9b9008a2361b, 3c52bfbcef564636ad56cbf737af90e0, b9ee55440e0046f6b8a2eb8aa0f040f7; no voters:
I20260807 18:07:16.766552 19664 raft_consensus.cc:2781] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:07:16.766713 19664 raft_consensus.cc:482] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:07:16.766820 19664 raft_consensus.cc:3037] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:16.769151 19664 raft_consensus.cc:504] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.769812 19664 leader_election.cc:288] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [CANDIDATE]: Term 1 election: Requested vote from peers 3c52bfbcef564636ad56cbf737af90e0 (127.8.62.130:38049), 07eca32d132b45e3803b9b9008a2361b (127.8.62.131:40221)
I20260807 18:07:16.770342 19593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793448e1de3b46cdaffe01d508e84da1" candidate_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07eca32d132b45e3803b9b9008a2361b"
I20260807 18:07:16.770330 19456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793448e1de3b46cdaffe01d508e84da1" candidate_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c52bfbcef564636ad56cbf737af90e0"
I20260807 18:07:16.770568 19593 raft_consensus.cc:3037] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:16.770623 19456 raft_consensus.cc:3037] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:07:16.773990 19456 raft_consensus.cc:2454] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ee55440e0046f6b8a2eb8aa0f040f7 in term 1.
I20260807 18:07:16.773991 19593 raft_consensus.cc:2454] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9ee55440e0046f6b8a2eb8aa0f040f7 in term 1.
I20260807 18:07:16.774516 19251 leader_election.cc:302] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07eca32d132b45e3803b9b9008a2361b, b9ee55440e0046f6b8a2eb8aa0f040f7; no voters:
I20260807 18:07:16.774905 19664 raft_consensus.cc:2781] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:07:16.775396 19664 raft_consensus.cc:686] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 1 LEADER]: Becoming Leader. State: Replica: b9ee55440e0046f6b8a2eb8aa0f040f7, State: Running, Role: LEADER
I20260807 18:07:16.775895 19664 consensus_queue.cc:237] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } }
I20260807 18:07:16.781838 19169 catalog_manager.cc:5495] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b9ee55440e0046f6b8a2eb8aa0f040f7 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9ee55440e0046f6b8a2eb8aa0f040f7" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 } health_report { overall_health: UNKNOWN } } }
I20260807 18:07:16.821979 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.816287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.8.62.129:40039)
0807 18:07:16.816511 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.816518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.816537 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:07:16.816821 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.816825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.816967 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.817125 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:07:16.817500 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.818429 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.819765 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.820181 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.820311 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.820880 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:07:16.821397 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:07:16.821563 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.821621 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.821700 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:07:16.830120 19456 raft_consensus.cc:1261] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 1 FOLLOWER]: Refusing update from remote peer b9ee55440e0046f6b8a2eb8aa0f040f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:07:16.830185 19593 raft_consensus.cc:1261] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 1 FOLLOWER]: Refusing update from remote peer b9ee55440e0046f6b8a2eb8aa0f040f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:07:16.831090 19669 consensus_queue.cc:1035] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c52bfbcef564636ad56cbf737af90e0" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:07:16.831424 19664 consensus_queue.cc:1035] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07eca32d132b45e3803b9b9008a2361b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:07:16.846583 19672 mvcc.cc:205] Tried to move back new op lower bound from 7315972246846644224 to 7315972246639366144. Current Snapshot: MvccSnapshot[applied={T|T < 7315972246846644224}]
I20260807 18:07:16.865521 19169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:07:16.865844 19169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:07:16.869620 19169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5af68823ab43411085fc442ccef4cf97: Sending DeleteTablet for 3 replicas of tablet 793448e1de3b46cdaffe01d508e84da1
I20260807 18:07:16.870824 19298 tablet_service.cc:1514] Processing DeleteTablet for tablet 793448e1de3b46cdaffe01d508e84da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:07:16 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:57026
I20260807 18:07:16.870958 19436 tablet_service.cc:1514] Processing DeleteTablet for tablet 793448e1de3b46cdaffe01d508e84da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:07:16 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40158
I20260807 18:07:16.870949 19573 tablet_service.cc:1514] Processing DeleteTablet for tablet 793448e1de3b46cdaffe01d508e84da1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:07:16 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:60048
I20260807 18:07:16.874581 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 19232
I20260807 18:07:16.883571 19685 tablet_replica.cc:330] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: stopping tablet replica
I20260807 18:07:16.884182 19685 raft_consensus.cc:2229] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:07:16.884863 19685 raft_consensus.cc:2258] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:07:16.885857 19684 tablet_replica.cc:330] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: stopping tablet replica
I20260807 18:07:16.886126 19683 tablet_replica.cc:330] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: stopping tablet replica
I20260807 18:07:16.886422 19684 raft_consensus.cc:2229] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:07:16.886708 19683 raft_consensus.cc:2229] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:07:16.886987 19684 raft_consensus.cc:2258] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:07:16.887526 19683 raft_consensus.cc:2258] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:07:16.889678 19683 ts_tablet_manager.cc:1905] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:07:16.889758 19684 ts_tablet_manager.cc:1905] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:07:16.897574 19684 ts_tablet_manager.cc:1918] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:07:16.897774 19684 log.cc:1196] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/793448e1de3b46cdaffe01d508e84da1
I20260807 18:07:16.897982 19683 ts_tablet_manager.cc:1918] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:07:16.898204 19683 log.cc:1196] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/793448e1de3b46cdaffe01d508e84da1
I20260807 18:07:16.898396 19684 ts_tablet_manager.cc:1939] T 793448e1de3b46cdaffe01d508e84da1 P 3c52bfbcef564636ad56cbf737af90e0: Deleting consensus metadata
I20260807 18:07:16.898898 19683 ts_tablet_manager.cc:1939] T 793448e1de3b46cdaffe01d508e84da1 P b9ee55440e0046f6b8a2eb8aa0f040f7: Deleting consensus metadata
I20260807 18:07:16.900704 19685 ts_tablet_manager.cc:1905] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:07:16.904353 19153 catalog_manager.cc:4841] TS 3c52bfbcef564636ad56cbf737af90e0 (127.8.62.130:38049): tablet 793448e1de3b46cdaffe01d508e84da1 (table test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]) successfully deleted
I20260807 18:07:16.904820 19153 catalog_manager.cc:4841] TS b9ee55440e0046f6b8a2eb8aa0f040f7 (127.8.62.129:40039): tablet 793448e1de3b46cdaffe01d508e84da1 (table test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]) successfully deleted
Aug 07 18:07:16 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:07:16.908033 19685 ts_tablet_manager.cc:1918] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:07:16.908275 19685 log.cc:1196] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/793448e1de3b46cdaffe01d508e84da1
I20260807 18:07:16.908921 19685 ts_tablet_manager.cc:1939] T 793448e1de3b46cdaffe01d508e84da1 P 07eca32d132b45e3803b9b9008a2361b: Deleting consensus metadata
I20260807 18:07:16.911021 19154 catalog_manager.cc:4841] TS 07eca32d132b45e3803b9b9008a2361b (127.8.62.131:40221): tablet 793448e1de3b46cdaffe01d508e84da1 (table test-table [id=fd6dcd1536ae47f78ea659dd5e925ac9]) successfully deleted
I20260807 18:07:16.914716 19653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:16.876625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57058 (local address 127.8.62.129:40039)
0807 18:07:16.876892 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:16.876901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:16.891524 (+ 14623us) server_negotiation.cc:407] Connection header received
0807 18:07:16.892028 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:16.892034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:16.892138 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:16.892216 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:16.894112 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.895461 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.897574 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:16.898428 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:16.898547 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:16.906381 (+ 7834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:16.906404 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:16.906424 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:16.906492 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:16.911109 (+ 4617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.912110 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.912121 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.912124 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.912357 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:16.912878 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:16.912883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:16.912890 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:16.913397 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:16.913706 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:16.914075 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 18:07:16.914347 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 18:07:16.917647 19358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:17.020009 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 19232
I20260807 18:07:17.039837 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 19369
Aug 07 18:07:17 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:07:17.056058 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:17.040774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40166 (local address 127.8.62.130:38049)
0807 18:07:17.041004 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:17.041010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:17.041033 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:07:17.041108 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:17.041113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:17.041203 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:17.041314 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:17.042578 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:17.043360 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:17.044697 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:17.045105 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:17.045244 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:17.049638 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:17.049656 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:17.049671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:17.049709 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:17.053084 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:17.053821 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:17.053848 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:17.053854 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:17.054004 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:17.054388 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:17.054393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:17.054396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:17.054809 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:17.055130 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:17.055459 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:07:17.055686 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:07:17.056923 19497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:17.150542 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 19369
I20260807 18:07:17.169480 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 19508
Aug 07 18:07:17 dist-test-slave-rnrw krb5kdc[19123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:07:17.186131 19655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:07:17.170356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.8.62.131:40221)
0807 18:07:17.170570 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 18:07:17.170586 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:07:17.170608 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:07:17.170721 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:07:17.170726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:07:17.170807 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:07:17.170919 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:07:17.172231 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:17.173045 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:17.174510 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:07:17.174925 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:07:17.175070 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:07:17.179816 (+ 4746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:07:17.179834 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:07:17.179851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:07:17.179893 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:07:17.182883 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:17.183696 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:17.183704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:17.183707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:17.183867 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:07:17.184283 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:07:17.184289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:07:17.184294 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:07:17.184874 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:07:17.185212 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 18:07:17.185519 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 18:07:17.185761 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:07:17.186851 19633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:17.277659 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 19508
I20260807 18:07:17.295730 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 19136
I20260807 18:07:17.296967 19200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:07:17.408977 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 19136
2026-08-07T18:07:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4625 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:07:17 dist-test-slave-rnrw krb5kdc[19694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:07:17 dist-test-slave-rnrw krb5kdc[19694](info): set up 2 sockets
Aug 07 18:07:17 dist-test-slave-rnrw krb5kdc[19694](info): commencing operation
krb5kdc: starting...
W20260807 18:07:18.555783 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:07:18 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:07:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangeradmin_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangeradmin_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangerlookup_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangerlookup_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
I20260807 18:07:18.920627 8442 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskn0cCOk/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/postgres -l logfile start
2026-08-07 18:07:21.811 UTC [19725] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:07:21.811 UTC [19725] LOG: listening on IPv4 address "127.8.62.148", port 56683
2026-08-07 18:07:21.815 UTC [19725] LOG: listening on Unix socket "/tmp/.s.PGSQL.56683"
2026-08-07 18:07:21.838 UTC [19728] LOG: database system was shut down at 2026-08-07 18:07:20 UTC
2026-08-07 18:07:21.842 UTC [19725] LOG: database system is ready to accept connections
I20260807 18:07:22.847502 8442 mini_postgres.cc:96] Postgres bound to 56683
2026-08-07 18:07:22.869 UTC [19736] FATAL: database "slave" does not exist
127.8.62.148:56683 - accepting connections
I20260807 18:07:22.870462 8442 mini_ranger.cc:161] Starting Ranger...
I20260807 18:07:22.901031 8442 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:07:23.495168 8442 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:07:23.495319 8442 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin
2026-08-07 18:07:24,005 [I] DB FLAVOR :POSTGRES
2026-08-07 18:07:24,006 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:07:24,006 [I] Checking connection..
2026-08-07 18:07:24,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:07:24,299 [I] Checking connection passed.
2026-08-07 18:07:24,299 [I] --------- Verifying version history table ---------
2026-08-07 18:07:24,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:07:24,565 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:07:24,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:07:24,835 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:07:24,836 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:07:24,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:07:25.086 UTC [19802] WARNING: there is no transaction in progress
2026-08-07 18:07:25,093 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:07:25,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:07:25,346 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:07:25,346 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:07:25,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:07:25,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:07:25,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rnrw.c.gcp-upstream.internal','N') ;"
2026-08-07 18:07:26,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:07:26,372 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:07:26,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:07:26,648 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:07:26,649 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:07:26,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:07:27,160 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:07:27,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:07:28.184 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.203 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.220 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.237 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.255 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.275 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.338 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.345 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.361 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.372 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.384 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.401 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.412 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.420 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.429 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:28.438 UTC [19913] WARNING: there is no transaction in progress
2026-08-07 18:07:29,163 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:07:29,163 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:07:29,404 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:07:29,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:07:29,681 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:07:29,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:07:29,953 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:07:29,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:07:30,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:07:30,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rnrw.c.gcp-upstream.internal';"
2026-08-07 18:07:30,759 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:07:30,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:07:31,007 [I] DB_PATCHES have already been applied
I20260807 18:07:31.014112 8442 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskn0cCOk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:07:31.014209 8442 mini_ranger.cc:193] Using host: 127.8.62.148
I20260807 18:07:31.031304 8442 mini_ranger.cc:227] Ranger admin URL: http://127.8.62.148:39599
Aug 07, 2026 6:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:07:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:07:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:07:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:07:33.381476 8442 mini_ranger.cc:160] Time spent starting Ranger: real 10.511s user 0.000s sys 0.058s
Aug 07, 2026 6:07:55 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:07:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 07, 2026 6:07:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:07:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:07:56 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:07:56.946543 8442 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangerkms_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/rangerkms_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:07:57 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126077, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:07:57.155496 8442 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:07:57.183179 8442 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:07:58.126370 8442 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:07:58.126573 8442 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms
I20260807 18:07:58.126695 8442 mini_ranger_kms.cc:227] Using postgres at 127.8.62.148:56683
2026-08-07 18:07:58,952 [I] DB FLAVOR :POSTGRES
2026-08-07 18:07:58,953 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:07:58,953 [I] Checking connection
2026-08-07 18:07:58,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:07:59,224 [I] connection success
2026-08-07 18:07:59,224 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:07:59,224 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:07:59,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:07:59,475 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:07:59,476 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:07:59,476 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:07:59,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:56683/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:07:59,746 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:08:00.274797 8442 mini_ranger_kms.cc:310] Created kms service
I20260807 18:08:00.495316 8442 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:08:00.544116 8442 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:08:00 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126077, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:00.991320 8442 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:08:00.991714 8442 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms:/tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:08:00.991780 8442 mini_ranger_kms.cc:240] Using host: 127.8.62.148
I20260807 18:08:01.016822 8442 mini_ranger_kms.cc:276] Ranger KMS PID: 20099
I20260807 18:08:01.017056 8442 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.62.148:51091
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:08:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 18:08:03.414119 8442 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.259s user 0.152s sys 0.415s
I20260807 18:08:03.414392 8442 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:08:03.414700 8442 mini_ranger_kms.cc:401] 127.8.62.148:51091/kms/v1/keys
Aug 07 18:08:03 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126083, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 18:08:04 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126083, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
Aug 07 18:08:04 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126083, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
Aug 07 18:08:07 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:08:07.145707 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35843
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35843
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51091/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:08:07.695979 20149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:08:07.696429 20149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:08:07.696651 20149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:08:07.706445 20149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:08:07.706696 20149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:08:07.706836 20149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:08:07.706945 20149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:08:07.707041 20149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:08:07.719879 20149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.62.148:51091/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35843
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:08:07.721484 20149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:08:07.722916 20149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:08:07.739609 20154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:07.739425 20155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:07.739425 20157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:08:07.740669 20149 server_base.cc:1034] running on GCE node
I20260807 18:08:07.741606 20149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:08:07.743211 20149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:08:07.744462 20149 hybrid_clock.cc:648] HybridClock initialized: now 1786126087744414 us; error 59 us; skew 500 ppm
I20260807 18:08:07.744939 20149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:08:07 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:08:07.753616 20149 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:08:07.755601 20149 webserver.cc:460] Webserver started at http://127.8.62.190:46097/ using document root <none> and password file <none>
I20260807 18:08:07.756318 20149 fs_manager.cc:359] Metadata directory not provided
I20260807 18:08:07.756456 20149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:08:07.756843 20149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:08:07 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:07.824970 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "54082b4a4762435cac5dfc5c7a3b5dbe"
format_stamp: "Formatted at 2026-08-07 18:08:07 on dist-test-slave-rnrw"
server_key: "157563ee4085fb739abb79e0e3246656"
server_key_iv: "0271fcee26fe135f6a216d415d2520b2"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:07.826223 20149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "54082b4a4762435cac5dfc5c7a3b5dbe"
format_stamp: "Formatted at 2026-08-07 18:08:07 on dist-test-slave-rnrw"
server_key: "157563ee4085fb739abb79e0e3246656"
server_key_iv: "0271fcee26fe135f6a216d415d2520b2"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:07.833410 20149 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 18:08:07.857120 20165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:07.858989 20149 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260807 18:08:07.859269 20149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "54082b4a4762435cac5dfc5c7a3b5dbe"
format_stamp: "Formatted at 2026-08-07 18:08:07 on dist-test-slave-rnrw"
server_key: "157563ee4085fb739abb79e0e3246656"
server_key_iv: "0271fcee26fe135f6a216d415d2520b2"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:07.859560 20149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:08:07.874791 20149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:08:07.880327 20149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:08:07.880879 20149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:08:07.918756 20149 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:35843
I20260807 18:08:07.918779 20216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:35843 every 8 connection(s)
I20260807 18:08:07.920532 20149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:08:07.921831 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 20149
I20260807 18:08:07.922044 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:08:07.922900 8442 external_mini_cluster.cc:1428] Setting key 3f5f49c46aafd159b09153cac90e4c7c
I20260807 18:08:07.927882 20217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:08:07.946321 20217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: Bootstrap starting.
I20260807 18:08:07.951203 20217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: Neither blocks nor log segments found. Creating new log.
I20260807 18:08:07.952756 20217 log.cc:824] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:08:07 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:08:07.957512 20217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: No bootstrap required, opened a new log
I20260807 18:08:07.967110 20220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:07.925902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38046 (local address 127.8.62.190:35843)
0807 18:08:07.933539 (+ 7637us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:07.933563 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:07.937496 (+ 3933us) server_negotiation.cc:407] Connection header received
0807 18:08:07.940755 (+ 3259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:07.940800 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:07.941485 (+ 685us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:07.942041 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:07.944055 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:07.946852 (+ 2797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:07.948941 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:07.950252 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:07.950463 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:07.955761 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:07.955774 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:07.955793 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:07.955843 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:07.960928 (+ 5085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:07.961951 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:07.961961 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:07.961971 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:07.962217 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:07.962762 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:07.962767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:07.962771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:07.963712 (+ 941us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:07.964014 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:07.964646 (+ 632us) server_negotiation.cc:299] Negotiation successful
0807 18:08:07.965202 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7239,"thread_start_us":6863,"threads_started":1}
I20260807 18:08:07.971427 20217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } }
I20260807 18:08:07.972008 20217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:08:07.972208 20217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54082b4a4762435cac5dfc5c7a3b5dbe, State: Initialized, Role: FOLLOWER
I20260807 18:08:07.972993 20217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } }
I20260807 18:08:07.973294 20217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:08:07.973500 20217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:08:07.973686 20217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:08:07.976804 20217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } }
I20260807 18:08:07.977502 20217 leader_election.cc:302] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54082b4a4762435cac5dfc5c7a3b5dbe; no voters:
I20260807 18:08:07.978677 20217 leader_election.cc:288] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:08:07.978775 20222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:08:07.980770 20222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [term 1 LEADER]: Becoming Leader. State: Replica: 54082b4a4762435cac5dfc5c7a3b5dbe, State: Running, Role: LEADER
I20260807 18:08:07.981679 20217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:08:07.981647 20222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } }
I20260807 18:08:07.990631 20223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } } }
I20260807 18:08:07.991379 20223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [sys.catalog]: This master's current role is: LEADER
I20260807 18:08:07.992455 20224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54082b4a4762435cac5dfc5c7a3b5dbe. Latest consensus state: current_term: 1 leader_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54082b4a4762435cac5dfc5c7a3b5dbe" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35843 } } }
I20260807 18:08:07.992933 20224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe [sys.catalog]: This master's current role is: LEADER
I20260807 18:08:08.002857 20228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:08:08.011057 20228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:08:08.040527 20228 catalog_manager.cc:1347] Generated new cluster ID: 1de9bfeb9f5944089f7c130f843ef176
I20260807 18:08:08.040791 20228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:08:08.050539 20217 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:08:08.102172 20228 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:08:08.104736 20228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:08:08.119920 20228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: Generated new TSK 0
I20260807 18:08:08.121490 20228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:08:08.982450 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:08:09 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:08:09.564663 20264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:08:09.565193 20264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:08:09.565465 20264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:08:09.577483 20264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:08:09.577764 20264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:08:09.578019 20264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:08:09.596964 20264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.62.148:51091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:08:09.598898 20264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:08:09.601087 20264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:08:09.625795 20274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:09.626772 20275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:09.628620 20278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:08:09.631722 20264 server_base.cc:1034] running on GCE node
I20260807 18:08:09.632907 20264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:08:09.634176 20264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:08:09.635565 20264 hybrid_clock.cc:648] HybridClock initialized: now 1786126089635380 us; error 117 us; skew 500 ppm
I20260807 18:08:09.636019 20264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:08:09 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:08:09.641077 20264 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:08:09.646353 20264 webserver.cc:460] Webserver started at http://127.8.62.129:33855/ using document root <none> and password file <none>
I20260807 18:08:09.647467 20264 fs_manager.cc:359] Metadata directory not provided
I20260807 18:08:09.647711 20264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:08:09.648139 20264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:08:09 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:09.671263 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "619bb25adaf6411a902531f1ada97584"
format_stamp: "Formatted at 2026-08-07 18:08:09 on dist-test-slave-rnrw"
server_key: "38005f5a5e60841b8c5c63d775a4d25b"
server_key_iv: "ce84cfad0c8534389e9455f4f46d8459"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:09.674942 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "619bb25adaf6411a902531f1ada97584"
format_stamp: "Formatted at 2026-08-07 18:08:09 on dist-test-slave-rnrw"
server_key: "38005f5a5e60841b8c5c63d775a4d25b"
server_key_iv: "ce84cfad0c8534389e9455f4f46d8459"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:09.695900 20264 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.011s sys 0.007s
I20260807 18:08:09.733302 20285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:09.738370 20264 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.006s sys 0.009s
I20260807 18:08:09.738826 20264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "619bb25adaf6411a902531f1ada97584"
format_stamp: "Formatted at 2026-08-07 18:08:09 on dist-test-slave-rnrw"
server_key: "38005f5a5e60841b8c5c63d775a4d25b"
server_key_iv: "ce84cfad0c8534389e9455f4f46d8459"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:09.739248 20264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 18:08:09 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:09.786370 20264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:08:09.793136 20264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:08:09.793807 20264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:08:09.795483 20264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:08:09.802906 20264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:08:09.803068 20264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:09.803421 20264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:08:09.803591 20264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:08:10 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:10.374198 20404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:37825 every 8 connection(s)
I20260807 18:08:10.374719 20264 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:37825
I20260807 18:08:10.377190 20264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:08:10.386161 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 20264
I20260807 18:08:10.386394 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:08:10.387310 8442 external_mini_cluster.cc:1428] Setting key 122a7570744aae31a67649fd5f8ef871
Aug 07 18:08:10 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126089, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:08:10.442082 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:10.388240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:39069 (local address 127.8.62.190:35843)
0807 18:08:10.412390 (+ 24150us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:10.412397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:10.416627 (+ 4230us) server_negotiation.cc:407] Connection header received
0807 18:08:10.419058 (+ 2431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:10.419065 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:10.419165 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:10.419246 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:10.421605 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:10.422766 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:10.425676 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:10.426439 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:10.426548 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:10.432652 (+ 6104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:10.432660 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:10.432665 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:10.432711 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:10.437386 (+ 4675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:10.438623 (+ 1237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:10.438630 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:10.438634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:10.438800 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:10.439562 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:10.439567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:10.439571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:10.439968 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:10.440113 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:10.441210 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0807 18:08:10.441468 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23947,"thread_start_us":25157,"threads_started":1}
I20260807 18:08:10.444981 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:10.385065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35843 (local address 127.8.62.129:39069)
0807 18:08:10.414576 (+ 29511us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:10.414632 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:10.418658 (+ 4026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:10.419831 (+ 1173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:10.419843 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:10.420509 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:08:10.421388 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:10.421415 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:10.422997 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:10.423003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:10.425482 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:10.425498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:10.427083 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:10.427092 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:10.427470 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:10.428476 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:08:10.428510 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:08:10.432288 (+ 3778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:08:10.437680 (+ 5392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:10.437691 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:10.437713 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:10.438451 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:10.438957 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:10.438962 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:10.438966 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:10.439403 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:10.440215 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:08:10.440223 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:08:10.440748 (+ 525us) client_negotiation.cc:769] Sending connection context
0807 18:08:10.442686 (+ 1938us) client_negotiation.cc:240] Negotiation successful
0807 18:08:10.443281 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29171,"thread_start_us":31554,"threads_started":1}
I20260807 18:08:10.447785 20405 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35843
I20260807 18:08:10.448177 20405 heartbeater.cc:464] Registering TS with master...
I20260807 18:08:10.449326 20405 heartbeater.cc:511] Master 127.8.62.190:35843 requested a full tablet report, sending...
I20260807 18:08:10.452410 20182 ts_manager.cc:194] Registered new tserver with Master: 619bb25adaf6411a902531f1ada97584 (127.8.62.129:37825)
I20260807 18:08:10.454864 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:39069
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
W20260807 18:08:10.527345 8442 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.62.130: real 0.136s user 0.000s sys 0.028s
I20260807 18:08:10.528363 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:08:11.157277 20411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:08:11.157604 20411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:08:11.157770 20411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:08:11.168454 20411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:08:11.168807 20411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:08:11.169064 20411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:08:11.181636 20411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.62.148:51091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:08:11.183485 20411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:08:11.185446 20411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:08:11.203363 20423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:11.203766 20424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:11.204135 20426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:08:11.205724 20411 server_base.cc:1034] running on GCE node
I20260807 18:08:11.206449 20411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:08:11.207619 20411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:08:11.208894 20411 hybrid_clock.cc:648] HybridClock initialized: now 1786126091208815 us; error 70 us; skew 500 ppm
I20260807 18:08:11.209287 20411 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:08:11 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:08:11.212720 20411 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:08:11.214490 20411 webserver.cc:460] Webserver started at http://127.8.62.130:42283/ using document root <none> and password file <none>
I20260807 18:08:11.215250 20411 fs_manager.cc:359] Metadata directory not provided
I20260807 18:08:11.215389 20411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:08:11.215735 20411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:08:11 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:11.233409 20411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "4d7cce68fe934cff95113aae461d2008"
format_stamp: "Formatted at 2026-08-07 18:08:11 on dist-test-slave-rnrw"
server_key: "9397bec65604a535878911967a03e8b1"
server_key_iv: "ce2f5a3422d57a6172c54621454c64af"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:11.234726 20411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "4d7cce68fe934cff95113aae461d2008"
format_stamp: "Formatted at 2026-08-07 18:08:11 on dist-test-slave-rnrw"
server_key: "9397bec65604a535878911967a03e8b1"
server_key_iv: "ce2f5a3422d57a6172c54621454c64af"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:11.243255 20411 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260807 18:08:11.260334 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:11.262428 20411 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:08:11.262704 20411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "4d7cce68fe934cff95113aae461d2008"
format_stamp: "Formatted at 2026-08-07 18:08:11 on dist-test-slave-rnrw"
server_key: "9397bec65604a535878911967a03e8b1"
server_key_iv: "ce2f5a3422d57a6172c54621454c64af"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:11.263190 20411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:08:11.279685 20411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:08:11.285207 20411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:08:11.285676 20411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:08:11.287228 20411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:08:11.289337 20411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:08:11.289476 20411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:11.289683 20411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:08:11.289774 20411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:11.352320 20411 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:45879
I20260807 18:08:11.352643 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:45879 every 8 connection(s)
I20260807 18:08:11.354015 20411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:08:11.355374 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 20411
I20260807 18:08:11.355597 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:08:11.356339 8442 external_mini_cluster.cc:1428] Setting key b9bd94ec7c2e8f1fada33bbc5029c29b
Aug 07 18:08:11 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:08:11.392663 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:11.359223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:52713 (local address 127.8.62.190:35843)
0807 18:08:11.373187 (+ 13964us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:11.373192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:11.373212 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:08:11.373271 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:11.373275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:11.373343 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:11.373624 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:11.376313 (+ 2689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:11.377337 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:11.379339 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:11.379758 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:11.379907 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:11.384995 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:11.385001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:11.385004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:11.385049 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:11.388469 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:11.389473 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:11.389478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:11.389484 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:11.389689 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:11.390180 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:11.390186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:11.390189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:11.390541 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:11.390796 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:11.391820 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0807 18:08:11.392064 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13826,"thread_start_us":13666,"threads_started":1}
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:08:11.393796 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:11.358267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35843 (local address 127.8.62.130:52713)
0807 18:08:11.364537 (+ 6270us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:11.364617 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:11.368981 (+ 4364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:11.373996 (+ 5015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:11.374014 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:11.375155 (+ 1141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:08:11.376081 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:11.376106 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:11.377520 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:11.377530 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:11.379161 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:11.379176 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:11.380078 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:11.380096 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:11.380452 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:11.381449 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:08:11.381475 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:08:11.384707 (+ 3232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:08:11.388708 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:11.388719 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:11.388743 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:11.389322 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:11.389834 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:11.389839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:11.389842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:11.390058 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:11.390731 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:08:11.390753 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:08:11.391314 (+ 561us) client_negotiation.cc:769] Sending connection context
0807 18:08:11.391649 (+ 335us) client_negotiation.cc:240] Negotiation successful
0807 18:08:11.392265 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5906,"thread_start_us":5645,"threads_started":1}
I20260807 18:08:11.395821 20550 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35843
I20260807 18:08:11.396261 20550 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
I20260807 18:08:11.397315 20550 heartbeater.cc:511] Master 127.8.62.190:35843 requested a full tablet report, sending...
I20260807 18:08:11.399101 20182 ts_manager.cc:194] Registered new tserver with Master: 4d7cce68fe934cff95113aae461d2008 (127.8.62.130:45879)
I20260807 18:08:11.400400 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:52713
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:08:11.432154 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51091/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:08:11.459208 20405 heartbeater.cc:503] Master 127.8.62.190:35843 was elected leader, sending a full tablet report...
W20260807 18:08:11.942582 20556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:08:11.942875 20556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:08:11.943077 20556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:08:11.953040 20556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:08:11.953157 20556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:08:11.953367 20556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:08:11.965118 20556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.62.148:51091/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:08:11.966610 20556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:08:11.968209 20556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:08:11.985898 20561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:11.986465 20564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:08:11.986994 20562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:08:11.987862 20556 server_base.cc:1034] running on GCE node
I20260807 18:08:11.988806 20556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:08:11.990056 20556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:08:11.991313 20556 hybrid_clock.cc:648] HybridClock initialized: now 1786126091991219 us; error 57 us; skew 500 ppm
I20260807 18:08:11.991781 20556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:08:11 dist-test-slave-rnrw krb5kdc[19694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:08:11.995967 20556 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:08:11.998175 20556 webserver.cc:460] Webserver started at http://127.8.62.131:33487/ using document root <none> and password file <none>
I20260807 18:08:11.999045 20556 fs_manager.cc:359] Metadata directory not provided
I20260807 18:08:11.999195 20556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:08:11.999591 20556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:08:11 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:12.018854 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "f58f0a7b84a24889aeed8c59ea863c35"
format_stamp: "Formatted at 2026-08-07 18:08:12 on dist-test-slave-rnrw"
server_key: "58773df25d22d4723e067165f5c7a9e9"
server_key_iv: "0774bd5054192fe08affadd1c2335565"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:12.019791 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "f58f0a7b84a24889aeed8c59ea863c35"
format_stamp: "Formatted at 2026-08-07 18:08:12 on dist-test-slave-rnrw"
server_key: "58773df25d22d4723e067165f5c7a9e9"
server_key_iv: "0774bd5054192fe08affadd1c2335565"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:12.026564 20556 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260807 18:08:12.046322 20573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:12.048101 20556 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 18:08:12.048331 20556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "f58f0a7b84a24889aeed8c59ea863c35"
format_stamp: "Formatted at 2026-08-07 18:08:12 on dist-test-slave-rnrw"
server_key: "58773df25d22d4723e067165f5c7a9e9"
server_key_iv: "0774bd5054192fe08affadd1c2335565"
server_key_version: "kuduclusterkey@0"
I20260807 18:08:12.048647 20556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:08:12.072352 20556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:08:12.077157 20556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:08:12.077653 20556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:08:12.079077 20556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:08:12.081262 20556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:08:12.081395 20556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:12.081609 20556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:08:12.081758 20556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:08:12.151962 20556 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:40999
I20260807 18:08:12.151999 20686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:40999 every 8 connection(s)
I20260807 18:08:12.154047 20556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:08:12.164434 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 20556
I20260807 18:08:12.164698 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:08:12.165216 8442 external_mini_cluster.cc:1428] Setting key 725d17d87708fe58142c5b4fdfed83c3
Aug 07 18:08:12 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126091, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:08:12.182826 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.157757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:45169 (local address 127.8.62.190:35843)
0807 18:08:12.158499 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.158505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.159800 (+ 1295us) server_negotiation.cc:407] Connection header received
0807 18:08:12.162427 (+ 2627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.162434 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.162533 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.162675 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:12.165044 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.166089 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.168694 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.169523 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.169683 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.174029 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:12.174036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:12.174040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:12.174088 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:12.177889 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.179085 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.179090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.179092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.179243 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.180125 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.180129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.180131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.180718 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:12.180869 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.182009 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.182369 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":471,"threads_started":1}
I20260807 18:08:12.184235 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.158315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35843 (local address 127.8.62.131:45169)
0807 18:08:12.159440 (+ 1125us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.159500 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.162097 (+ 2597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.162950 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.162959 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.163733 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:08:12.164826 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.164860 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.166263 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.166268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.168385 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.168406 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.169856 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.169866 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.170123 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.170958 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:08:12.170991 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:08:12.173655 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:08:12.178088 (+ 4433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:12.178098 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:12.178124 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:12.178863 (+ 739us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:12.179392 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:08:12.179396 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:08:12.179399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:08:12.179946 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:08:12.180915 (+ 969us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:08:12.180924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:08:12.181523 (+ 599us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.181804 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.182525 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":874,"thread_start_us":546,"threads_started":1}
I20260807 18:08:12.185977 20687 heartbeater.cc:347] Connected to a master server at 127.8.62.190:35843
I20260807 18:08:12.186465 20687 heartbeater.cc:464] Registering TS with master...
I20260807 18:08:12.187664 20687 heartbeater.cc:511] Master 127.8.62.190:35843 requested a full tablet report, sending...
I20260807 18:08:12.189462 20182 ts_manager.cc:194] Registered new tserver with Master: f58f0a7b84a24889aeed8c59ea863c35 (127.8.62.131:40999)
I20260807 18:08:12.190375 20182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:45169
I20260807 18:08:12.196090 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:08:12.219405 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.202200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.8.62.190:35843)
0807 18:08:12.202410 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.202427 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.203290 (+ 863us) server_negotiation.cc:407] Connection header received
0807 18:08:12.203482 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.203487 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.203626 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.203725 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:12.205823 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.206781 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.208678 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.209303 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.209419 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.212446 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:12.212451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:12.212455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:12.212494 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:12.216165 (+ 3671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.217188 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.217193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.217195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.217414 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.217922 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.217926 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.217928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.218233 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:12.218441 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.218877 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.219090 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:08:12.224848 20182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:08:12.234999 20182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:08:12.277799 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.251470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49466 (local address 127.8.62.131:40999)
0807 18:08:12.264882 (+ 13412us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.264897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.266714 (+ 1817us) server_negotiation.cc:407] Connection header received
0807 18:08:12.266807 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.266813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.267196 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.267445 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:08:12.269220 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.270927 (+ 1707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.275053 (+ 4126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.276271 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.276451 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.276824 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.277130 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.277337 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13118,"thread_start_us":14337,"threads_started":1}
I20260807 18:08:12.277933 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.251216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:40999 (local address 127.0.0.1:49466)
0807 18:08:12.266155 (+ 14939us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.266179 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.266371 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.268049 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.268053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.268075 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:08:12.268934 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.268952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.271066 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.271075 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.274803 (+ 3728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.274816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.276609 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.276617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.276843 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.276908 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.277049 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.277279 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14720,"mutex_wait_us":90,"thread_start_us":11029,"threads_started":1}
I20260807 18:08:12.278764 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.250871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:45879 (local address 127.0.0.1:46710)
0807 18:08:12.263056 (+ 12185us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.263146 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.266227 (+ 3081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.267673 (+ 1446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.267683 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.267734 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:08:12.269160 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.269173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.271544 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.271550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.274757 (+ 3207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.274773 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.277448 (+ 2675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.277457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.277961 (+ 504us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.277987 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.278054 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.278214 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11720,"mutex_wait_us":133,"thread_start_us":16293,"threads_started":1}
I20260807 18:08:12.279037 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.264228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.8.62.129:37825)
0807 18:08:12.266635 (+ 2407us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.266644 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.267185 (+ 541us) server_negotiation.cc:407] Connection header received
0807 18:08:12.267881 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.267889 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.268137 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.268316 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:08:12.269796 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.271868 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.274867 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.276766 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.276928 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.277289 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.278186 (+ 897us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.278403 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2135,"thread_start_us":1861,"threads_started":1}
I20260807 18:08:12.279641 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.251746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.8.62.130:45879)
0807 18:08:12.265737 (+ 13991us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.265747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.265780 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:08:12.266634 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.266643 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.266963 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.267199 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:08:12.269326 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.271388 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.274924 (+ 3536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.276768 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.276928 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.277139 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.279082 (+ 1943us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.279302 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13821,"thread_start_us":13605,"threads_started":1}
I20260807 18:08:12.281278 20700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.263464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:37825 (local address 127.0.0.1:35842)
0807 18:08:12.267057 (+ 3593us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.267078 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.267636 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.268407 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.268411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.268440 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:08:12.269530 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.269546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.272055 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.272059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.274656 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.274669 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.277584 (+ 2915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.277591 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.277922 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.277966 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.278055 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.280710 (+ 2655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3421,"mutex_wait_us":2649,"thread_start_us":2190,"threads_started":1}
I20260807 18:08:12.283344 20621 tablet_service.cc:1467] Processing CreateTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 (DEFAULT_TABLE table=test-table [id=19ee1d96a7ab472e9303eec08618b540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:08:12.284144 20483 tablet_service.cc:1467] Processing CreateTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 (DEFAULT_TABLE table=test-table [id=19ee1d96a7ab472e9303eec08618b540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:08:12.285483 20621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c3f82d514ae48bf96f88199fa7d4a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:08:12.286047 20483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c3f82d514ae48bf96f88199fa7d4a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:08:12.289094 20334 tablet_service.cc:1467] Processing CreateTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 (DEFAULT_TABLE table=test-table [id=19ee1d96a7ab472e9303eec08618b540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:08:12.291365 20334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c3f82d514ae48bf96f88199fa7d4a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:08:12.306977 20708 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Bootstrap starting.
I20260807 18:08:12.310456 20709 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Bootstrap starting.
I20260807 18:08:12.311923 20710 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Bootstrap starting.
I20260807 18:08:12.314363 20708 tablet_bootstrap.cc:654] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Neither blocks nor log segments found. Creating new log.
I20260807 18:08:12.317103 20708 log.cc:824] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Log is configured to *not* fsync() on all Append() calls
I20260807 18:08:12.318441 20710 tablet_bootstrap.cc:654] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Neither blocks nor log segments found. Creating new log.
I20260807 18:08:12.318483 20709 tablet_bootstrap.cc:654] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Neither blocks nor log segments found. Creating new log.
I20260807 18:08:12.320251 20709 log.cc:824] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Log is configured to *not* fsync() on all Append() calls
I20260807 18:08:12.320363 20710 log.cc:824] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Log is configured to *not* fsync() on all Append() calls
I20260807 18:08:12.320931 20708 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: No bootstrap required, opened a new log
I20260807 18:08:12.321449 20708 ts_tablet_manager.cc:1397] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.010s
I20260807 18:08:12.324306 20710 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: No bootstrap required, opened a new log
I20260807 18:08:12.324756 20710 ts_tablet_manager.cc:1397] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260807 18:08:12.326797 20709 tablet_bootstrap.cc:492] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: No bootstrap required, opened a new log
I20260807 18:08:12.327241 20709 ts_tablet_manager.cc:1397] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20260807 18:08:12.334829 20710 raft_consensus.cc:348] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.335419 20710 raft_consensus.cc:374] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:08:12.335644 20710 raft_consensus.cc:729] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619bb25adaf6411a902531f1ada97584, State: Initialized, Role: FOLLOWER
I20260807 18:08:12.335815 20708 raft_consensus.cc:348] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.336373 20710 consensus_queue.cc:260] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.336665 20708 raft_consensus.cc:374] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:08:12.336838 20708 raft_consensus.cc:729] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d7cce68fe934cff95113aae461d2008, State: Initialized, Role: FOLLOWER
I20260807 18:08:12.337777 20708 consensus_queue.cc:260] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.339273 20710 ts_tablet_manager.cc:1428] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 18:08:12.341005 20550 heartbeater.cc:503] Master 127.8.62.190:35843 was elected leader, sending a full tablet report...
I20260807 18:08:12.342437 20708 ts_tablet_manager.cc:1428] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Time spent starting tablet: real 0.020s user 0.006s sys 0.012s
I20260807 18:08:12.342193 20709 raft_consensus.cc:348] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.342732 20709 raft_consensus.cc:374] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:08:12.342936 20709 raft_consensus.cc:729] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58f0a7b84a24889aeed8c59ea863c35, State: Initialized, Role: FOLLOWER
I20260807 18:08:12.343648 20709 consensus_queue.cc:260] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.345916 20687 heartbeater.cc:503] Master 127.8.62.190:35843 was elected leader, sending a full tablet report...
I20260807 18:08:12.346645 20709 ts_tablet_manager.cc:1428] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Time spent starting tablet: real 0.019s user 0.014s sys 0.007s
I20260807 18:08:12.353976 20716 raft_consensus.cc:482] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:08:12.354323 20716 raft_consensus.cc:504] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.370946 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.358631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:55831 (local address 127.8.62.129:37825)
0807 18:08:12.358937 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.358945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.358971 (+ 26us) server_negotiation.cc:407] Connection header received
0807 18:08:12.359080 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.359087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.359189 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.359304 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:08:12.360218 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.362145 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.366052 (+ 3907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.368329 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.368572 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.368712 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.370238 (+ 1526us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.370463 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:08:12.371411 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.356656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:37825 (local address 127.8.62.131:55831)
0807 18:08:12.357132 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.357157 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.357280 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.359572 (+ 2292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.359581 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.359630 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:08:12.360001 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.360017 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.362386 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.362395 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.365785 (+ 3390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.365809 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.368861 (+ 3052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.368871 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.369296 (+ 425us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.369357 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.369538 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.369766 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311}
I20260807 18:08:12.372604 20716 leader_election.cc:288] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d7cce68fe934cff95113aae461d2008 (127.8.62.130:45879), 619bb25adaf6411a902531f1ada97584 (127.8.62.129:37825)
I20260807 18:08:12.372880 20356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c3f82d514ae48bf96f88199fa7d4a62" candidate_uuid: "f58f0a7b84a24889aeed8c59ea863c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619bb25adaf6411a902531f1ada97584" is_pre_election: true
I20260807 18:08:12.373907 20356 raft_consensus.cc:2454] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58f0a7b84a24889aeed8c59ea863c35 in term 0.
I20260807 18:08:12.374722 20576 leader_election.cc:302] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619bb25adaf6411a902531f1ada97584, f58f0a7b84a24889aeed8c59ea863c35; no voters:
I20260807 18:08:12.375242 20716 raft_consensus.cc:2781] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:08:12.375434 20716 raft_consensus.cc:482] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:08:12.375640 20716 raft_consensus.cc:3037] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:08:12.376011 20551 tablet.cc:2369] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:08:12.380028 20716 raft_consensus.cc:504] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.380936 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.357867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:37513 (local address 127.8.62.130:45879)
0807 18:08:12.358138 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.358146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.372060 (+ 13914us) server_negotiation.cc:407] Connection header received
0807 18:08:12.372483 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.372489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.372712 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.372860 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:08:12.373887 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.375137 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.378675 (+ 3538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.379696 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.379821 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.379875 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.380579 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.380728 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:08:12.381162 20716 leader_election.cc:288] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [CANDIDATE]: Term 1 election: Requested vote from peers 4d7cce68fe934cff95113aae461d2008 (127.8.62.130:45879), 619bb25adaf6411a902531f1ada97584 (127.8.62.129:37825)
I20260807 18:08:12.381193 20690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.356865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:45879 (local address 127.8.62.131:37513)
0807 18:08:12.371952 (+ 15087us) negotiation.cc:107] Waiting for socket to connect
0807 18:08:12.371977 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:08:12.372288 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:08:12.373149 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:08:12.373165 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:08:12.373205 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:08:12.373719 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.373739 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.375279 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.375287 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.378417 (+ 3130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:08:12.378453 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.379940 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.379953 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:08:12.380336 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.380371 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 18:08:12.380514 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 18:08:12.380714 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14917,"thread_start_us":16476,"threads_started":1}
I20260807 18:08:12.382068 20356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c3f82d514ae48bf96f88199fa7d4a62" candidate_uuid: "f58f0a7b84a24889aeed8c59ea863c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619bb25adaf6411a902531f1ada97584"
I20260807 18:08:12.382400 20356 raft_consensus.cc:3037] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:08:12.383358 20502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c3f82d514ae48bf96f88199fa7d4a62" candidate_uuid: "f58f0a7b84a24889aeed8c59ea863c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7cce68fe934cff95113aae461d2008"
I20260807 18:08:12.383421 20503 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c3f82d514ae48bf96f88199fa7d4a62" candidate_uuid: "f58f0a7b84a24889aeed8c59ea863c35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d7cce68fe934cff95113aae461d2008" is_pre_election: true
I20260807 18:08:12.384364 20502 raft_consensus.cc:3037] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:08:12.386588 20356 raft_consensus.cc:2454] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58f0a7b84a24889aeed8c59ea863c35 in term 1.
I20260807 18:08:12.387181 20576 leader_election.cc:302] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619bb25adaf6411a902531f1ada97584, f58f0a7b84a24889aeed8c59ea863c35; no voters:
I20260807 18:08:12.387632 20716 raft_consensus.cc:2781] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:08:12.388237 20502 raft_consensus.cc:2454] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58f0a7b84a24889aeed8c59ea863c35 in term 1.
I20260807 18:08:12.388907 20716 raft_consensus.cc:686] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 1 LEADER]: Becoming Leader. State: Replica: f58f0a7b84a24889aeed8c59ea863c35, State: Running, Role: LEADER
I20260807 18:08:12.389474 20716 consensus_queue.cc:237] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } }
I20260807 18:08:12.396153 20182 catalog_manager.cc:5495] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 reported cstate change: term changed from 0 to 1, leader changed from <none> to f58f0a7b84a24889aeed8c59ea863c35 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "f58f0a7b84a24889aeed8c59ea863c35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f58f0a7b84a24889aeed8c59ea863c35" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 } health_report { overall_health: UNKNOWN } } }
W20260807 18:08:12.406972 20688 tablet.cc:2369] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:08:12.415400 20408 tablet.cc:2369] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:08:12.423700 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.416472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.8.62.131:40999)
0807 18:08:12.416731 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.416736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.416753 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:08:12.416845 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.416848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.416923 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.417052 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:08:12.417608 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.418917 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.420989 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.421564 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.421671 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.422300 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:08:12.422890 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:08:12.423017 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.423183 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.423271 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 18:08:12.434516 20502 raft_consensus.cc:1261] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 1 FOLLOWER]: Refusing update from remote peer f58f0a7b84a24889aeed8c59ea863c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:08:12.435021 20356 raft_consensus.cc:1261] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 1 FOLLOWER]: Refusing update from remote peer f58f0a7b84a24889aeed8c59ea863c35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:08:12.435786 20718 consensus_queue.cc:1035] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d7cce68fe934cff95113aae461d2008" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 45879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 18:08:12.436820 20716 consensus_queue.cc:1035] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619bb25adaf6411a902531f1ada97584" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 37825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:08:12.473956 20181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:08:12.474260 20181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:08:12.478888 20181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54082b4a4762435cac5dfc5c7a3b5dbe: Sending DeleteTablet for 3 replicas of tablet 9c3f82d514ae48bf96f88199fa7d4a62
I20260807 18:08:12.480626 20621 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:08:12 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:49466
I20260807 18:08:12.480659 20483 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:08:12 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:46710
I20260807 18:08:12.481387 20334 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c3f82d514ae48bf96f88199fa7d4a62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:08:12 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:35842
I20260807 18:08:12.482966 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 20264
I20260807 18:08:12.497306 20733 tablet_replica.cc:330] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: stopping tablet replica
I20260807 18:08:12.498086 20733 raft_consensus.cc:2229] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:08:12.498683 20732 tablet_replica.cc:330] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: stopping tablet replica
I20260807 18:08:12.498911 20733 raft_consensus.cc:2258] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:08:12.499413 20734 tablet_replica.cc:330] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: stopping tablet replica
I20260807 18:08:12.499991 20732 raft_consensus.cc:2229] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:08:12.500501 20734 raft_consensus.cc:2229] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:08:12.501260 20732 raft_consensus.cc:2258] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:08:12.501567 20733 ts_tablet_manager.cc:1905] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:08:12.502276 20734 raft_consensus.cc:2258] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:08:12.504752 20732 ts_tablet_manager.cc:1905] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:08:12.504667 20734 ts_tablet_manager.cc:1905] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:08:12.510552 20733 ts_tablet_manager.cc:1918] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:08:12.510865 20733 log.cc:1196] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/9c3f82d514ae48bf96f88199fa7d4a62
Aug 07 18:08:12 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:08:12.511663 20733 ts_tablet_manager.cc:1939] T 9c3f82d514ae48bf96f88199fa7d4a62 P 4d7cce68fe934cff95113aae461d2008: Deleting consensus metadata
I20260807 18:08:12.513988 20732 ts_tablet_manager.cc:1918] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:08:12.514276 20732 log.cc:1196] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/9c3f82d514ae48bf96f88199fa7d4a62
I20260807 18:08:12.514204 20167 catalog_manager.cc:4841] TS 4d7cce68fe934cff95113aae461d2008 (127.8.62.130:45879): tablet 9c3f82d514ae48bf96f88199fa7d4a62 (table test-table [id=19ee1d96a7ab472e9303eec08618b540]) successfully deleted
I20260807 18:08:12.515122 20732 ts_tablet_manager.cc:1939] T 9c3f82d514ae48bf96f88199fa7d4a62 P f58f0a7b84a24889aeed8c59ea863c35: Deleting consensus metadata
I20260807 18:08:12.517287 20168 catalog_manager.cc:4841] TS f58f0a7b84a24889aeed8c59ea863c35 (127.8.62.131:40999): tablet 9c3f82d514ae48bf96f88199fa7d4a62 (table test-table [id=19ee1d96a7ab472e9303eec08618b540]) successfully deleted
I20260807 18:08:12.519284 20734 ts_tablet_manager.cc:1918] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:08:12.519481 20734 log.cc:1196] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/9c3f82d514ae48bf96f88199fa7d4a62
I20260807 18:08:12.520143 20734 ts_tablet_manager.cc:1939] T 9c3f82d514ae48bf96f88199fa7d4a62 P 619bb25adaf6411a902531f1ada97584: Deleting consensus metadata
I20260807 18:08:12.522084 20168 catalog_manager.cc:4841] TS 619bb25adaf6411a902531f1ada97584 (127.8.62.129:37825): tablet 9c3f82d514ae48bf96f88199fa7d4a62 (table test-table [id=19ee1d96a7ab472e9303eec08618b540]) successfully deleted
I20260807 18:08:12.525049 20707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.484271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35856 (local address 127.8.62.129:37825)
0807 18:08:12.484468 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.484474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.489203 (+ 4729us) server_negotiation.cc:407] Connection header received
0807 18:08:12.489448 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.489454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.489543 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.489664 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:12.493708 (+ 4044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.495257 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.498537 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.499315 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.499424 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.515537 (+ 16113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:12.515555 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:12.515575 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:12.515630 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:12.521414 (+ 5784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.522647 (+ 1233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.522654 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.522656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.522826 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.523363 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.523367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.523369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.523844 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:12.524122 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.524523 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.524773 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 18:08:12.526156 20400 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:08:12.623947 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 20264
I20260807 18:08:12.653226 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 20411
Aug 07 18:08:12 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:08:12.677085 20704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.654549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46724 (local address 127.8.62.130:45879)
0807 18:08:12.654775 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.654783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.654807 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:08:12.654878 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.654886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.655010 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.655126 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:12.657137 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.658395 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.660570 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.661284 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.661434 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.668419 (+ 6985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:12.668444 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:12.668493 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:12.668589 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:12.673358 (+ 4769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.674321 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.674332 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.674337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.674518 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.675037 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.675043 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.675046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.675564 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:12.675894 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.676262 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.676524 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:08:12.677800 20545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:08:12.790690 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 20411
I20260807 18:08:12.811300 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 20556
Aug 07 18:08:12 dist-test-slave-rnrw krb5kdc[19694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:08:12.835667 20703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:08:12.812166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49476 (local address 127.8.62.131:40999)
0807 18:08:12.812372 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:08:12.812378 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:08:12.812401 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:08:12.812572 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:08:12.812578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:08:12.812668 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:08:12.812827 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:08:12.814288 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.815488 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.817125 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:08:12.817874 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:08:12.818038 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:08:12.828009 (+ 9971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:08:12.828032 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:08:12.828052 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:08:12.828106 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:08:12.832185 (+ 4079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.833083 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.833094 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.833097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.833366 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:08:12.833801 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:08:12.833807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:08:12.833825 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:08:12.834387 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:08:12.834724 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:08:12.834987 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 18:08:12.835241 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:08:12.836687 20682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:08:12.965977 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 20556
I20260807 18:08:12.987608 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 20149
I20260807 18:08:12.988912 20212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:08:13.104923 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 20149
I20260807 18:08:13.133272 8442 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:08:13.479202 8442 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:08:13.479470 8442 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:08:13.825757 8442 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:08:13.826 UTC [19725] LOG: received smart shutdown request
2026-08-07 18:08:13.830 UTC [19725] LOG: background worker "logical replication launcher" (PID 19734) exited with exit code 1
2026-08-07 18:08:13.830 UTC [19729] LOG: shutting down
2026-08-07 18:08:13.982 UTC [19725] LOG: database system is shut down
2026-08-07T18:08:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56704 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:08:14 dist-test-slave-rnrw krb5kdc[20748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:08:14 dist-test-slave-rnrw krb5kdc[20748](info): set up 2 sockets
Aug 07 18:08:14 dist-test-slave-rnrw krb5kdc[20748](info): commencing operation
krb5kdc: starting...
W20260807 18:08:15.263825 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:08:15 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:08:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:08:15Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangeradmin_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangeradmin_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangerlookup_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangerlookup_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
I20260807 18:08:15.639431 8442 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskn0cCOk/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/postgres -l logfile start
2026-08-07 18:08:18.374 UTC [20779] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 18:08:18.375 UTC [20779] LOG: listening on IPv4 address "127.8.62.148", port 41073
2026-08-07 18:08:18.379 UTC [20779] LOG: listening on Unix socket "/tmp/.s.PGSQL.41073"
2026-08-07 18:08:18.403 UTC [20782] LOG: database system was shut down at 2026-08-07 18:08:16 UTC
2026-08-07 18:08:18.407 UTC [20779] LOG: database system is ready to accept connections
I20260807 18:08:19.410900 8442 mini_postgres.cc:96] Postgres bound to 41073
2026-08-07 18:08:19.432 UTC [20790] FATAL: database "slave" does not exist
127.8.62.148:41073 - accepting connections
I20260807 18:08:19.433365 8442 mini_ranger.cc:161] Starting Ranger...
I20260807 18:08:19.463070 8442 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 18:08:19.989777 8442 mini_ranger.cc:87] Created ranger Postgres database
I20260807 18:08:19.989915 8442 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin
2026-08-07 18:08:20,277 [I] DB FLAVOR :POSTGRES
2026-08-07 18:08:20,277 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:08:20,278 [I] Checking connection..
2026-08-07 18:08:20,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 18:08:20,537 [I] Checking connection passed.
2026-08-07 18:08:20,537 [I] --------- Verifying version history table ---------
2026-08-07 18:08:20,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:08:20,822 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:08:20,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:08:21,098 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 18:08:21,099 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 18:08:21,099 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 18:08:21.373 UTC [20856] WARNING: there is no transaction in progress
2026-08-07 18:08:21,381 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 18:08:21,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 18:08:21,664 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 18:08:21,664 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:08:21,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 18:08:21,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 18:08:22,204 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-rnrw.c.gcp-upstream.internal','N') ;"
2026-08-07 18:08:22,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:08:22,747 [I] Table x_portal_user does not exist in database ranger
2026-08-07 18:08:22,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:08:23,036 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 18:08:23,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:08:23,311 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:08:23,563 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 18:08:23,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 18:08:24.640 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.659 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.678 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.694 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.715 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.737 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.806 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.813 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.829 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.840 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.851 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.868 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.880 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.889 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.897 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:24.906 UTC [20967] WARNING: there is no transaction in progress
2026-08-07 18:08:25,624 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 18:08:25,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 18:08:25,873 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 18:08:25,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 18:08:26,136 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 18:08:26,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 18:08:26,394 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 18:08:26,395 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 18:08:26,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 18:08:26,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-rnrw.c.gcp-upstream.internal';"
2026-08-07 18:08:27,141 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 18:08:27,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 18:08:27,413 [I] DB_PATCHES have already been applied
I20260807 18:08:27.419548 8442 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskn0cCOk/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 18:08:27.419637 8442 mini_ranger.cc:193] Using host: 127.8.62.148
I20260807 18:08:27.436079 8442 mini_ranger.cc:227] Ranger admin URL: http://127.8.62.148:56277
Aug 07, 2026 6:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 6:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 6:08:27 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 6:08:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 6:08:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 18:08:28.694012 8442 mini_ranger.cc:160] Time spent starting Ranger: real 9.261s user 0.000s sys 0.044s
Aug 07, 2026 6:08:49 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 6:08:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 6:08:49 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 6:08:49 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 6:08:50 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 18:08:50.697834 8442 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.62.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.62.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangerkms_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.62.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/rangerkms_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.62.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.62.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.62.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/HTTP_127.8.62.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 18:08:50 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126130, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 18:08:50.918696 8442 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 18:08:50.946543 8442 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 18:08:51.964591 8442 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 18:08:51.964704 8442 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms
I20260807 18:08:51.964754 8442 mini_ranger_kms.cc:227] Using postgres at 127.8.62.148:41073
2026-08-07 18:08:52,350 [I] DB FLAVOR :POSTGRES
2026-08-07 18:08:52,350 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 18:08:52,350 [I] Checking connection
2026-08-07 18:08:52,350 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 18:08:52,628 [I] connection success
2026-08-07 18:08:52,629 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 18:08:52,629 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 18:08:52,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 18:08:52,944 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 18:08:52,944 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 18:08:52,944 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 18:08:52,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.62.148:41073/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 18:08:53,242 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 18:08:53.735435 8442 mini_ranger_kms.cc:310] Created kms service
I20260807 18:08:53.962425 8442 mini_ranger_kms.cc:326] Created kudu user
I20260807 18:08:54.011961 8442 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 18:08:54 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126130, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:08:54.502625 8442 mini_ranger_kms.cc:384] Added ranger policy
I20260807 18:08:54.502975 8442 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms:/tmp/dist-test-taskn0cCOk/build/asan/bin/postgresql.jar:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskn0cCOk/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 18:08:54.503024 8442 mini_ranger_kms.cc:240] Using host: 127.8.62.148
I20260807 18:08:54.523430 8442 mini_ranger_kms.cc:276] Ranger KMS PID: 21153
I20260807 18:08:54.523607 8442 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.62.148:51871
Aug 07, 2026 6:08:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 6:08:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 6:08:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 6:08:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 6:08:56 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 6:08:56 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 18:08:56.942489 8442 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.024s user 0.064s sys 0.342s
I20260807 18:08:56.942739 8442 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 18:08:56.942901 8442 mini_ranger_kms.cc:401] 127.8.62.148:51871/kms/v1/keys
Aug 07 18:08:57 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 18:08:57 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
Aug 07 18:08:57 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126137, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.62.148@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
Aug 07 18:09:00 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:09:00.441435 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46373
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46373
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:00.808907 21205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:00.809396 21205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:00.809563 21205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:00.820632 21205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:00.820788 21205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:00.820853 21205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:00.820931 21205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:00.821004 21205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:00.836714 21205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.62.148:51871/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46373
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:00.838510 21205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:00.840226 21205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:00.854358 21211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:00.854866 21213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:00.854923 21210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:00.855805 21205 server_base.cc:1034] running on GCE node
I20260807 18:09:00.856938 21205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:00.858907 21205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:00.860181 21205 hybrid_clock.cc:648] HybridClock initialized: now 1786126140860114 us; error 74 us; skew 500 ppm
I20260807 18:09:00.860635 21205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:00 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:00.870375 21205 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:00.872246 21205 webserver.cc:460] Webserver started at http://127.8.62.190:39507/ using document root <none> and password file <none>
I20260807 18:09:00.873063 21205 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:00.873204 21205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:00.873560 21205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:09:00 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:00.939766 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "e486b80cea7e45b5a5260fe24a02c71f"
format_stamp: "Formatted at 2026-08-07 18:09:00 on dist-test-slave-rnrw"
server_key: "611cccbcf903eac4738702b7c50b2812"
server_key_iv: "a1cdd9a098eecaca73e241f0160da001"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:00.940919 21205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "e486b80cea7e45b5a5260fe24a02c71f"
format_stamp: "Formatted at 2026-08-07 18:09:00 on dist-test-slave-rnrw"
server_key: "611cccbcf903eac4738702b7c50b2812"
server_key_iv: "a1cdd9a098eecaca73e241f0160da001"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:00.946753 21205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:09:00.967172 21220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:00.968894 21205 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:00.969111 21205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "e486b80cea7e45b5a5260fe24a02c71f"
format_stamp: "Formatted at 2026-08-07 18:09:00 on dist-test-slave-rnrw"
server_key: "611cccbcf903eac4738702b7c50b2812"
server_key_iv: "a1cdd9a098eecaca73e241f0160da001"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:00.969364 21205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:00.992406 21205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:00.996984 21205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:00.997351 21205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:01.019394 21205 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46373
I20260807 18:09:01.019397 21272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46373 every 8 connection(s)
I20260807 18:09:01.021104 21205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:01.022853 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21205
I20260807 18:09:01.023106 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:09:01.023769 8442 external_mini_cluster.cc:1428] Setting key 4b36e696d329c0ee59ad289def210238
I20260807 18:09:01.026400 21273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:09:01 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:01.042068 21273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Bootstrap starting.
I20260807 18:09:01.046257 21273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:01.047713 21276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:01.026153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.8.62.190:46373)
0807 18:09:01.027074 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:01.027088 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:01.027132 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:09:01.028965 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:01.028989 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:01.029393 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:01.029859 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:01.031369 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:01.032856 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:01.034325 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:01.035051 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:01.035266 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:01.039408 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:01.039420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:01.039437 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:01.039481 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:01.042806 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:01.043595 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:01.043604 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:01.043613 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:01.043875 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:01.044453 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:01.044460 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:01.044463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:01.045164 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:01.045469 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:01.045966 (+ 497us) server_negotiation.cc:299] Negotiation successful
0807 18:09:01.046453 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":516,"threads_started":1}
I20260807 18:09:01.047856 21273 log.cc:824] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:01.051358 21273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: No bootstrap required, opened a new log
I20260807 18:09:01.059211 21273 raft_consensus.cc:348] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:01.059737 21273 raft_consensus.cc:374] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:01.059865 21273 raft_consensus.cc:729] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e486b80cea7e45b5a5260fe24a02c71f, State: Initialized, Role: FOLLOWER
I20260807 18:09:01.060534 21273 consensus_queue.cc:260] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:01.060837 21273 raft_consensus.cc:388] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:01.060981 21273 raft_consensus.cc:482] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:01.061170 21273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:01.063369 21273 raft_consensus.cc:504] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:01.063911 21273 leader_election.cc:302] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e486b80cea7e45b5a5260fe24a02c71f; no voters:
I20260807 18:09:01.064602 21273 leader_election.cc:288] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:09:01.064738 21278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:01.066011 21278 raft_consensus.cc:686] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 LEADER]: Becoming Leader. State: Replica: e486b80cea7e45b5a5260fe24a02c71f, State: Running, Role: LEADER
I20260807 18:09:01.066725 21278 consensus_queue.cc:237] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:01.067034 21273 sys_catalog.cc:564] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:01.073925 21281 sys_catalog.cc:455] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e486b80cea7e45b5a5260fe24a02c71f. Latest consensus state: current_term: 1 leader_uuid: "e486b80cea7e45b5a5260fe24a02c71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } } }
I20260807 18:09:01.074456 21281 sys_catalog.cc:458] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:01.075369 21280 sys_catalog.cc:455] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e486b80cea7e45b5a5260fe24a02c71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } } }
I20260807 18:09:01.075770 21280 sys_catalog.cc:458] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:01.082753 21289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:01.089291 21289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:01.092986 21273 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:09:01.131137 21289 catalog_manager.cc:1347] Generated new cluster ID: 9b86cc2ab1f4442d95524f3c586bb80a
I20260807 18:09:01.131306 21289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:01.157277 21289 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:09:01.159832 21289 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:01.173640 21289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Generated new TSK 0
I20260807 18:09:01.175225 21289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:09:01.977072 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:09:02 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 18:09:02.411487 21319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:02.411959 21319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:02.412209 21319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:02.424398 21319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:02.424607 21319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:02.424870 21319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:02.436179 21319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:02.437966 21319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:02.439647 21319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:02.457985 21329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:02.465500 21330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:02.465844 21332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:02.466765 21319 server_base.cc:1034] running on GCE node
I20260807 18:09:02.467676 21319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:02.469122 21319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:02.470491 21319 hybrid_clock.cc:648] HybridClock initialized: now 1786126142470418 us; error 58 us; skew 500 ppm
I20260807 18:09:02.470958 21319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:02 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:02.477389 21319 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:02.482357 21319 webserver.cc:460] Webserver started at http://127.8.62.129:42203/ using document root <none> and password file <none>
I20260807 18:09:02.483520 21319 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:02.483711 21319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:02.484217 21319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:09:02 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:02.515311 21319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "1b235d05782d42b59e1a92d081a2f6fe"
format_stamp: "Formatted at 2026-08-07 18:09:02 on dist-test-slave-rnrw"
server_key: "e0d08c61a0bebe539bbaf10091f9d09b"
server_key_iv: "39b88c69f29e62a848f54b1fe5cf25c1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:02.516784 21319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "1b235d05782d42b59e1a92d081a2f6fe"
format_stamp: "Formatted at 2026-08-07 18:09:02 on dist-test-slave-rnrw"
server_key: "e0d08c61a0bebe539bbaf10091f9d09b"
server_key_iv: "39b88c69f29e62a848f54b1fe5cf25c1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:02.538714 21319 fs_manager.cc:691] Time spent creating directory manager: real 0.021s user 0.015s sys 0.001s
I20260807 18:09:02.567739 21339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:02.569926 21319 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260807 18:09:02.570242 21319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "1b235d05782d42b59e1a92d081a2f6fe"
format_stamp: "Formatted at 2026-08-07 18:09:02 on dist-test-slave-rnrw"
server_key: "e0d08c61a0bebe539bbaf10091f9d09b"
server_key_iv: "39b88c69f29e62a848f54b1fe5cf25c1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:02.570664 21319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:02.629343 21319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:02.635438 21319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:02.635883 21319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:02.643678 21319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:02.653223 21319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:02.653431 21319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 18:09:02.653751 21319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:02.653909 21319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 18:09:02 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
Aug 07 18:09:03 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:03.345568 21456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:41649 every 8 connection(s)
I20260807 18:09:03.345980 21319 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:41649
I20260807 18:09:03.347898 21319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:03.349350 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21319
I20260807 18:09:03.349565 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:09:03.350358 8442 external_mini_cluster.cc:1428] Setting key cafaa64b8a949479b190db2abbd3fab1
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Aug 07 18:09:03 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126142, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:09:03.408206 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:03.367367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:47499 (local address 127.8.62.190:46373)
0807 18:09:03.384833 (+ 17466us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:03.384840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:03.384865 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:09:03.384940 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:03.384946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:03.385035 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:03.385177 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:03.388697 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:03.389872 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:03.393326 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:03.394000 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:03.396685 (+ 2685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:03.399096 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:03.399105 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:03.399113 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:03.399171 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:03.403507 (+ 4336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:03.404738 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:03.404747 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:03.404753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:03.404942 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:03.405592 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:03.405600 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:03.405606 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:03.406053 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:03.406190 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:03.407339 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0807 18:09:03.407663 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17308,"thread_start_us":17908,"threads_started":1}
I20260807 18:09:03.410784 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:03.366564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.129:47499)
0807 18:09:03.367945 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:03.367995 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:03.371360 (+ 3365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:03.385625 (+ 14265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:03.385642 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:03.386393 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:03.387388 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:03.387421 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:03.390137 (+ 2716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:03.390355 (+ 218us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:03.393074 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:03.393096 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:03.394321 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:03.394332 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:03.394658 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:03.395613 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:03.395649 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:03.398649 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:03.403742 (+ 5093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:03.403756 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:03.403782 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:03.404570 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:03.405115 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:03.405122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:03.405129 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:03.405437 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:03.406311 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:03.406322 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:03.406856 (+ 534us) client_negotiation.cc:769] Sending connection context
0807 18:09:03.408490 (+ 1634us) client_negotiation.cc:240] Negotiation successful
0807 18:09:03.409187 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":865,"threads_started":1}
I20260807 18:09:03.414003 21457 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:03.414393 21457 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:03.415390 21457 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:03.418289 21238 ts_manager.cc:194] Registered new tserver with Master: 1b235d05782d42b59e1a92d081a2f6fe (127.8.62.129:41649)
I20260807 18:09:03.420385 21238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:47499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:09:03.444576 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:03.890729 21464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:03.891142 21464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:03.891320 21464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:03.899742 21464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:03.899945 21464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:03.900156 21464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:03.912412 21464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:03.914004 21464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:03.915530 21464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:03.931293 21477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:03.931293 21478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:03.932103 21480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:03.932992 21464 server_base.cc:1034] running on GCE node
I20260807 18:09:03.933689 21464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:03.934854 21464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:03.936088 21464 hybrid_clock.cc:648] HybridClock initialized: now 1786126143936035 us; error 48 us; skew 500 ppm
I20260807 18:09:03.936444 21464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:03 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:03.939847 21464 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:03.941731 21464 webserver.cc:460] Webserver started at http://127.8.62.130:39739/ using document root <none> and password file <none>
I20260807 18:09:03.942485 21464 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:03.942629 21464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:03.942962 21464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:09:03 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:03.962571 21464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "ad2defaf947d4ecea526c982d38c8abb"
format_stamp: "Formatted at 2026-08-07 18:09:03 on dist-test-slave-rnrw"
server_key: "aed3e8553c83151362148c1e518a2eaa"
server_key_iv: "df4c341685a26ef76e94c80810d605d0"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:03.963804 21464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "ad2defaf947d4ecea526c982d38c8abb"
format_stamp: "Formatted at 2026-08-07 18:09:03 on dist-test-slave-rnrw"
server_key: "aed3e8553c83151362148c1e518a2eaa"
server_key_iv: "df4c341685a26ef76e94c80810d605d0"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:03.971199 21464 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 18:09:03.993235 21487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:03.995011 21464 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.007s sys 0.000s
I20260807 18:09:03.995272 21464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "ad2defaf947d4ecea526c982d38c8abb"
format_stamp: "Formatted at 2026-08-07 18:09:03 on dist-test-slave-rnrw"
server_key: "aed3e8553c83151362148c1e518a2eaa"
server_key_iv: "df4c341685a26ef76e94c80810d605d0"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:03.995523 21464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:04.020252 21464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:04.025578 21464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:04.025915 21464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:04.027364 21464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:04.029256 21464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:04.029357 21464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:04.029464 21464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:04.029521 21464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:04.065434 21464 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:46601
I20260807 18:09:04.065475 21603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:46601 every 8 connection(s)
I20260807 18:09:04.066812 21464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:04.074177 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21464
I20260807 18:09:04.074380 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:09:04.074901 8442 external_mini_cluster.cc:1428] Setting key 84f9c27f16a93f39483ea6347ba00480
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126143, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:04.088591 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.069649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:34173 (local address 127.8.62.190:46373)
0807 18:09:04.070124 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:04.070129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:04.071158 (+ 1029us) server_negotiation.cc:407] Connection header received
0807 18:09:04.073092 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:04.073100 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:04.073189 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:04.073348 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:04.075217 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.076041 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.078011 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.078490 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.078642 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.082186 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:04.082191 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:04.082194 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:04.082234 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:04.085253 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.086037 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.086041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.086044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.086170 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.086673 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.086677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.086679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.086972 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:04.087152 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:04.087920 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 18:09:04.088174 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260807 18:09:04.089653 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.070232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.130:34173)
0807 18:09:04.070988 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:04.071026 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:04.072722 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:04.073628 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:04.073647 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:04.074300 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:04.075030 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:04.075059 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.076206 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.076211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:04.077848 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:04.077869 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.078738 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.078746 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:04.079003 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.079744 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:04.079772 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:04.081940 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:04.085407 (+ 3467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:04.085416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:04.085435 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:04.085890 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:04.086295 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:04.086299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:04.086303 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:04.086519 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:04.087140 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:04.087149 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:04.087530 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 18:09:04.087819 (+ 289us) client_negotiation.cc:240] Negotiation successful
0807 18:09:04.088380 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":324,"threads_started":1}
I20260807 18:09:04.091353 21604 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:04.091714 21604 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:04.092512 21604 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:04.094228 21238 ts_manager.cc:194] Registered new tserver with Master: ad2defaf947d4ecea526c982d38c8abb (127.8.62.130:46601)
I20260807 18:09:04.095093 21238 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:34173
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:09:04.132336 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:04.425390 21457 heartbeater.cc:503] Master 127.8.62.190:46373 was elected leader, sending a full tablet report...
W20260807 18:09:04.484833 21610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:04.485141 21610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:04.485296 21610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:04.493057 21610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:04.493193 21610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:04.493412 21610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:04.503844 21610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:04.505303 21610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:04.506757 21610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:04.522074 21618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:04.522112 21615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:04.522222 21616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:04.526525 21610 server_base.cc:1034] running on GCE node
I20260807 18:09:04.527295 21610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:04.528487 21610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:04.529808 21610 hybrid_clock.cc:648] HybridClock initialized: now 1786126144529759 us; error 42 us; skew 500 ppm
I20260807 18:09:04.530206 21610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:04.533769 21610 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:04.535454 21610 webserver.cc:460] Webserver started at http://127.8.62.131:46741/ using document root <none> and password file <none>
I20260807 18:09:04.536188 21610 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:04.536293 21610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:04.536616 21610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:04.554410 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "dcc71732eab04b619e585a1aa9ad9c7f"
format_stamp: "Formatted at 2026-08-07 18:09:04 on dist-test-slave-rnrw"
server_key: "787f529f0814bafa0e5cf9adc0b89bb1"
server_key_iv: "592e73a0d034f5de87121e7e1b4998f1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:04.555480 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "dcc71732eab04b619e585a1aa9ad9c7f"
format_stamp: "Formatted at 2026-08-07 18:09:04 on dist-test-slave-rnrw"
server_key: "787f529f0814bafa0e5cf9adc0b89bb1"
server_key_iv: "592e73a0d034f5de87121e7e1b4998f1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:04.562155 21610 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:09:04.579118 21625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:04.581092 21610 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:04.581338 21610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "dcc71732eab04b619e585a1aa9ad9c7f"
format_stamp: "Formatted at 2026-08-07 18:09:04 on dist-test-slave-rnrw"
server_key: "787f529f0814bafa0e5cf9adc0b89bb1"
server_key_iv: "592e73a0d034f5de87121e7e1b4998f1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:04.581637 21610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:04.604108 21610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:04.608825 21610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:04.609149 21610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:04.610260 21610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:04.611727 21610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:04.611841 21610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:04.611999 21610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:04.612075 21610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:04.649209 21610 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:40917
I20260807 18:09:04.649267 21740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:40917 every 8 connection(s)
I20260807 18:09:04.650652 21610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:04.660064 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21610
I20260807 18:09:04.660291 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:09:04.660885 8442 external_mini_cluster.cc:1428] Setting key 525578b5223e90d02476d387ea92b19b
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126144, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:04.673087 21743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.653485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:40649 (local address 127.8.62.190:46373)
0807 18:09:04.654054 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:04.654058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:04.655120 (+ 1062us) server_negotiation.cc:407] Connection header received
0807 18:09:04.657152 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:04.657157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:04.657256 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:04.657362 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:04.658960 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.659817 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.661795 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.662489 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.662603 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.666714 (+ 4111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:04.666720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:04.666723 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:04.666765 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:04.669822 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.670623 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.670628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.670631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.670775 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.671237 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.671241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.671244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.671565 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:04.671698 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:04.672426 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 18:09:04.672721 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":329,"threads_started":1}
I20260807 18:09:04.674274 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.654024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.131:40649)
0807 18:09:04.654934 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:04.654981 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:04.656770 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:04.657617 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:04.657626 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:04.658093 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:04.658749 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:04.658768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.659982 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.659985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:04.661619 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:04.661637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.662795 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.662804 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:04.663121 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.663886 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:04.663937 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:04.666423 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:04.669983 (+ 3560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:04.669991 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:04.670006 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:04.670464 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:04.670878 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:04.670882 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:04.670884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:04.671096 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:04.671684 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:04.671691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:04.672026 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 18:09:04.672278 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 18:09:04.672772 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":367,"threads_started":1}
I20260807 18:09:04.675884 21741 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:04.676276 21741 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:04.677266 21741 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:04.679016 21237 ts_manager.cc:194] Registered new tserver with Master: dcc71732eab04b619e585a1aa9ad9c7f (127.8.62.131:40917)
I20260807 18:09:04.680119 21237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:40649
I20260807 18:09:04.689882 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:09:04.691694 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21319
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:09:04.712237 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.692838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.8.62.129:41649)
0807 18:09:04.693690 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:04.693698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:04.693723 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:09:04.693783 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:04.693790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:04.694008 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:04.694253 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:04.695997 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.697450 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.698921 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.699654 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.699798 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.704864 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:04.704883 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:04.704901 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:04.704955 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:04.708885 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.709784 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.709792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.709795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.709976 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.710456 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.710460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.710462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.710850 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:04.711114 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:04.711515 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 18:09:04.711762 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":415,"threads_started":1}
I20260807 18:09:04.713785 21452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:04.803884 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21319
I20260807 18:09:04.825165 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21464
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:09:04.844033 21748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.826242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.8.62.130:46601)
0807 18:09:04.826827 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:04.826835 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:04.826862 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:09:04.826924 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:04.826930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:04.827176 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:04.827389 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:04.828943 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.829992 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.831369 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.831959 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.832107 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.836899 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:04.836912 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:04.836929 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:04.836970 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:04.840307 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.841373 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.841382 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.841384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.841621 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.842183 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.842190 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.842195 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.842744 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:04.843023 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:04.843406 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 18:09:04.843713 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":331,"threads_started":1}
I20260807 18:09:04.845741 21599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:04.932346 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21464
I20260807 18:09:04.951426 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21610
Aug 07 18:09:04 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:09:04.970364 21750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:04.952475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.8.62.131:40917)
0807 18:09:04.953135 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:04.953143 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:04.953172 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:09:04.953234 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:04.953240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:04.953464 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:04.953661 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:04.955136 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.956190 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.957688 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:04.958479 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:04.958610 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:04.963216 (+ 4606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:04.963231 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:04.963249 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:04.963296 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:04.966583 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.967755 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.967764 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.967766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.968008 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:04.968461 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:04.968464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:04.968467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:04.968966 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:04.969230 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:04.969602 (+ 372us) server_negotiation.cc:299] Negotiation successful
0807 18:09:04.969836 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":340,"threads_started":1}
I20260807 18:09:04.971813 21736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:05.057256 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21610
I20260807 18:09:05.075544 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21205
I20260807 18:09:05.076974 21268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:05.190276 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21205
I20260807 18:09:05.217167 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46373
--webserver_interface=127.8.62.190
--webserver_port=39507
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46373
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:05.580936 21753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:05.581265 21753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:05.581482 21753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:05.589214 21753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:05.589344 21753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:05.589428 21753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:05.589520 21753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:05.589564 21753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:05.600960 21753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46373
--ranger_config_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=39507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:05.602454 21753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:05.603950 21753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:05.617681 21758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:05.617614 21759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:05.617645 21761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:05.618884 21753 server_base.cc:1034] running on GCE node
I20260807 18:09:05.619907 21753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:05.621645 21753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:05.622887 21753 hybrid_clock.cc:648] HybridClock initialized: now 1786126145622818 us; error 53 us; skew 500 ppm
I20260807 18:09:05.623373 21753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:05 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:05.627328 21753 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:05.629266 21753 webserver.cc:460] Webserver started at http://127.8.62.190:39507/ using document root <none> and password file <none>
I20260807 18:09:05.630021 21753 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:05.630158 21753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:09:05 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126145, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:05.681788 21753 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:09:05.685781 21770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:05.687651 21753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260807 18:09:05.687884 21753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "e486b80cea7e45b5a5260fe24a02c71f"
format_stamp: "Formatted at 2026-08-07 18:09:00 on dist-test-slave-rnrw"
server_key: "611cccbcf903eac4738702b7c50b2812"
server_key_iv: "a1cdd9a098eecaca73e241f0160da001"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:05.689330 21753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:05.704447 21753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:05.708796 21753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:05.709093 21753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:05.730578 21753 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46373
I20260807 18:09:05.730682 21822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46373 every 8 connection(s)
I20260807 18:09:05.732333 21753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:05.734850 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21753
I20260807 18:09:05.735913 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:41649
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:46373
--webserver_port=42203
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:05.742835 21823 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:09:05.747341 21823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Bootstrap starting.
I20260807 18:09:05.768347 21823 log.cc:824] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:05.778030 21823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:09:05.778630 21823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Bootstrap complete.
I20260807 18:09:05.788043 21823 raft_consensus.cc:348] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:05.788954 21823 raft_consensus.cc:729] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e486b80cea7e45b5a5260fe24a02c71f, State: Initialized, Role: FOLLOWER
I20260807 18:09:05.789634 21823 consensus_queue.cc:260] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:05.789871 21823 raft_consensus.cc:388] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:05.789991 21823 raft_consensus.cc:482] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:05.790187 21823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:09:05.794395 21823 raft_consensus.cc:504] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:05.794961 21823 leader_election.cc:302] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e486b80cea7e45b5a5260fe24a02c71f; no voters:
I20260807 18:09:05.795799 21823 leader_election.cc:288] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:09:05.795966 21827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:09:05.798363 21823 sys_catalog.cc:564] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:05.800815 21827 raft_consensus.cc:686] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [term 2 LEADER]: Becoming Leader. State: Replica: e486b80cea7e45b5a5260fe24a02c71f, State: Running, Role: LEADER
I20260807 18:09:05.801817 21827 consensus_queue.cc:237] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } }
I20260807 18:09:05.805064 21829 sys_catalog.cc:455] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e486b80cea7e45b5a5260fe24a02c71f. Latest consensus state: current_term: 2 leader_uuid: "e486b80cea7e45b5a5260fe24a02c71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } } }
I20260807 18:09:05.805064 21828 sys_catalog.cc:455] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e486b80cea7e45b5a5260fe24a02c71f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e486b80cea7e45b5a5260fe24a02c71f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46373 } } }
I20260807 18:09:05.805543 21829 sys_catalog.cc:458] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:05.805624 21828 sys_catalog.cc:458] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:05.817584 21840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:05.818213 21823 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 18:09:05.839805 21840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:05.841876 21840 catalog_manager.cc:1259] Loaded cluster ID: 9b86cc2ab1f4442d95524f3c586bb80a
I20260807 18:09:05.842075 21840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:05.847249 21840 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:05.849277 21840 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Loaded TSK: 0
I20260807 18:09:05.851194 21840 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:09:06.197145 21825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:06.197741 21825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:06.198005 21825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:06.212097 21825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:06.212387 21825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:06.212703 21825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:06.229705 21825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:41649
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=42203
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:06.231546 21825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:06.233516 21825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:06.260136 21861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:06.260952 21860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:06.270478 21863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:06.271476 21825 server_base.cc:1034] running on GCE node
I20260807 18:09:06.272241 21825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:06.273516 21825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:06.274806 21825 hybrid_clock.cc:648] HybridClock initialized: now 1786126146274735 us; error 68 us; skew 500 ppm
I20260807 18:09:06.275240 21825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:06 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126146, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:06.285442 21825 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:06.292855 21825 webserver.cc:460] Webserver started at http://127.8.62.129:42203/ using document root <none> and password file <none>
I20260807 18:09:06.294147 21825 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:06.294442 21825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:09:06 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126146, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:06.328743 21825 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.010s sys 0.004s
I20260807 18:09:06.332427 21871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:06.335704 21825 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260807 18:09:06.336028 21825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "1b235d05782d42b59e1a92d081a2f6fe"
format_stamp: "Formatted at 2026-08-07 18:09:02 on dist-test-slave-rnrw"
server_key: "e0d08c61a0bebe539bbaf10091f9d09b"
server_key_iv: "39b88c69f29e62a848f54b1fe5cf25c1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:06.337297 21825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:06.407394 21825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:06.412087 21825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:06.412447 21825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:06.413849 21825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:06.415439 21825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:06.415570 21825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:06.415707 21825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:06.415789 21825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:06.471206 21825 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:41649
I20260807 18:09:06.471230 21989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:41649 every 8 connection(s)
I20260807 18:09:06.473599 21825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:06.480229 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21825
I20260807 18:09:06.481138 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:46601
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:46373
--webserver_port=39739
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:09:06 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126146, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:06.553859 21993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:06.480380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.129:48823)
0807 18:09:06.516127 (+ 35747us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:06.516196 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:06.521056 (+ 4860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:06.531103 (+ 10047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:06.531116 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:06.531827 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:06.532744 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:06.532780 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:06.535359 (+ 2579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:06.535365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:06.537535 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:06.537554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:06.539488 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:06.539497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:06.539776 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:06.540645 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:06.540686 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:06.543858 (+ 3172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:06.547924 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:06.547938 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:06.547968 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:06.548752 (+ 784us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:06.549366 (+ 614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:06.549374 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:06.549377 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:06.549702 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:06.550828 (+ 1126us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:06.550839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:06.551354 (+ 515us) client_negotiation.cc:769] Sending connection context
0807 18:09:06.551730 (+ 376us) client_negotiation.cc:240] Negotiation successful
0807 18:09:06.552431 (+ 701us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35362,"thread_start_us":34269,"threads_started":1}
I20260807 18:09:06.553877 21992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:06.481533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:48823 (local address 127.8.62.190:46373)
0807 18:09:06.515155 (+ 33622us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:06.515179 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:06.518325 (+ 3146us) server_negotiation.cc:407] Connection header received
0807 18:09:06.529996 (+ 11671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:06.530013 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:06.530465 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:06.530911 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:06.533042 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:06.535113 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:06.537732 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:06.538824 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:06.538996 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:06.544229 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:06.544241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:06.544265 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:06.544326 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:06.547739 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:06.548958 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:06.548969 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:06.548977 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:06.549214 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:06.549863 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:06.549868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:06.549871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:06.550654 (+ 783us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:06.550900 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:06.552016 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0807 18:09:06.552471 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33176,"thread_start_us":33566,"threads_started":1}
I20260807 18:09:06.557785 21990 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:06.558152 21990 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:06.559186 21990 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
W20260807 18:09:06.562615 21990 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:46373 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:09:06.562847 21990 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:06.563090 21990 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:06.564307 21990 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:06.564695 21990 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
W20260807 18:09:06.565835 21990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:09:06.957485 21994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:06.957871 21994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:06.958036 21994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:06.967309 21994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:06.967446 21994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:06.967630 21994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:06.979243 21994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:46601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=39739
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:06.980938 21994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:06.982364 21994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:07.000875 22009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:07.001358 22008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:07.002951 21994 server_base.cc:1034] running on GCE node
W20260807 18:09:07.005795 22011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:07.006814 21994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:07.008330 21994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:07.009668 21994 hybrid_clock.cc:648] HybridClock initialized: now 1786126147009632 us; error 75 us; skew 500 ppm
I20260807 18:09:07.010123 21994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:07.014721 21994 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:07.016952 21994 webserver.cc:460] Webserver started at http://127.8.62.130:39739/ using document root <none> and password file <none>
I20260807 18:09:07.017792 21994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:07.017966 21994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:07.045698 21994 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:09:07.054283 22019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.056371 21994 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.003s sys 0.001s
I20260807 18:09:07.056682 21994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "ad2defaf947d4ecea526c982d38c8abb"
format_stamp: "Formatted at 2026-08-07 18:09:03 on dist-test-slave-rnrw"
server_key: "aed3e8553c83151362148c1e518a2eaa"
server_key_iv: "df4c341685a26ef76e94c80810d605d0"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:07.058264 21994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:07.121413 21994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:07.127297 21994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:07.127698 21994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:07.129134 21994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:07.131136 21994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:07.131269 21994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.131398 21994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:07.131487 21994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.183418 21994 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:46601
I20260807 18:09:07.183497 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:46601 every 8 connection(s)
I20260807 18:09:07.185626 21994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:07.188607 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 21994
I20260807 18:09:07.189430 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.62.148:51871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:40917
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:46373
--webserver_port=46741
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:07.223194 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.190370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:51109 (local address 127.8.62.190:46373)
0807 18:09:07.198809 (+ 8439us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:07.198816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:07.198845 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:09:07.201054 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:07.201060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:07.201171 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:07.201296 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:07.204316 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.205559 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.208328 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.209078 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.209195 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.214372 (+ 5177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:07.214379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:07.214385 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:07.214432 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:07.218755 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.219948 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.219954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.219957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.220151 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.220865 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.220870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.220873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.221272 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:07.221405 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:07.222506 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0807 18:09:07.222792 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8292,"thread_start_us":8133,"threads_started":1}
I20260807 18:09:07.226150 22139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.189669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.130:51109)
0807 18:09:07.196905 (+ 7236us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:07.196985 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:07.200435 (+ 3450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:07.201857 (+ 1422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:07.201871 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:07.202758 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:07.204002 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:07.204042 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.205823 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.205828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:07.208096 (+ 2268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:07.208126 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.209637 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.209645 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:07.209968 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.210958 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:07.210994 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:07.214016 (+ 3022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:07.218993 (+ 4977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:07.219005 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:07.219027 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:07.219767 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:07.220327 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:07.220332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:07.220336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:07.220698 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:07.221519 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:07.221542 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:07.222011 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 18:09:07.223838 (+ 1827us) client_negotiation.cc:240] Negotiation successful
0807 18:09:07.224580 (+ 742us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2297,"thread_start_us":2063,"threads_started":1}
I20260807 18:09:07.228482 22137 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:07.228856 22137 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:07.229789 22137 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:07.232097 21787 ts_manager.cc:194] Registered new tserver with Master: ad2defaf947d4ecea526c982d38c8abb (127.8.62.130:46601)
I20260807 18:09:07.234792 21787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:51109
I20260807 18:09:07.566020 21990 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:07.566453 21990 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:07.567898 21787 ts_manager.cc:194] Registered new tserver with Master: 1b235d05782d42b59e1a92d081a2f6fe (127.8.62.129:41649)
I20260807 18:09:07.569039 21787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:48823
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
W20260807 18:09:07.682520 22141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:07.683022 22141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:07.683218 22141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:07.694741 22141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:07.695006 22141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:07.695238 22141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:07.711009 22141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.62.148:51871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:40917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=46741
--tserver_master_addrs=127.8.62.190:46373
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:07.713069 22141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:07.714900 22141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:07.772331 22148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:07.772307 22147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:07.781147 22150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:07.782066 22141 server_base.cc:1034] running on GCE node
I20260807 18:09:07.782955 22141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:07.784343 22141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:07.785652 22141 hybrid_clock.cc:648] HybridClock initialized: now 1786126147785569 us; error 78 us; skew 500 ppm
I20260807 18:09:07.786072 22141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:07.790560 22141 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:07.794387 22141 webserver.cc:460] Webserver started at http://127.8.62.131:46741/ using document root <none> and password file <none>
I20260807 18:09:07.795463 22141 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:07.795674 22141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:07.820942 22141 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:09:07.825266 22158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.827407 22141 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:09:07.827734 22141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "dcc71732eab04b619e585a1aa9ad9c7f"
format_stamp: "Formatted at 2026-08-07 18:09:04 on dist-test-slave-rnrw"
server_key: "787f529f0814bafa0e5cf9adc0b89bb1"
server_key_iv: "592e73a0d034f5de87121e7e1b4998f1"
server_key_version: "kuduclusterkey@0"
I20260807 18:09:07.829458 22141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:07.849406 22141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:07.854082 22141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:07.854416 22141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:07.855921 22141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:07.857609 22141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:07.857738 22141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.857885 22141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:07.857982 22141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:07.901722 22141 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:40917
I20260807 18:09:07.901739 22273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:40917 every 8 connection(s)
I20260807 18:09:07.903650 22141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:07.913283 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22141
Aug 07 18:09:07 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:07.951002 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.914647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.8.62.190:46373)
0807 18:09:07.930513 (+ 15866us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:07.930519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:07.930546 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:09:07.930612 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:07.930618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:07.930723 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:07.930840 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:07.933276 (+ 2436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.934503 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.936602 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.937262 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.937392 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.941806 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:07.941814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:07.941819 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:07.941863 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:07.946300 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.947488 (+ 1188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.947496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.947499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.947688 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.948193 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.948198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.948202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.948631 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:07.948890 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:07.950298 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0807 18:09:07.950569 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":15721,"thread_start_us":15578,"threads_started":1}
I20260807 18:09:07.951712 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.908040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44601 (local address 127.8.62.190:46373)
0807 18:09:07.922010 (+ 13970us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:07.922021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:07.927310 (+ 5289us) server_negotiation.cc:407] Connection header received
0807 18:09:07.931053 (+ 3743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:07.931058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:07.931127 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:07.931199 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:07.933728 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.935010 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.937577 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.938252 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.938346 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.943183 (+ 4837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:07.943191 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:07.943196 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:07.943272 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:07.947259 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.948394 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.948399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.948403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.948663 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.949281 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.949286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.949290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.949645 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:07.949776 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:07.950858 (+ 1082us) server_negotiation.cc:299] Negotiation successful
0807 18:09:07.951089 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":13770,"spinlock_wait_cycles":1920,"thread_start_us":13992,"threads_started":1}
I20260807 18:09:07.954411 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.908953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46373 (local address 127.8.62.131:44601)
0807 18:09:07.927128 (+ 18175us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:07.927184 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:07.929789 (+ 2605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:07.931846 (+ 2057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:07.931859 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:07.932487 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:07.933431 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:07.933461 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.935211 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.935217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:07.937364 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:07.937390 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.938720 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.938726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:07.939045 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.939797 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:07.939828 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:07.942819 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:07.947494 (+ 4675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:07.947504 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:07.947538 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:07.948238 (+ 700us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:07.948829 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:07.948834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:07.948838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:07.949138 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:07.949888 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:07.949896 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:07.950359 (+ 463us) client_negotiation.cc:769] Sending connection context
0807 18:09:07.952216 (+ 1857us) client_negotiation.cc:240] Negotiation successful
0807 18:09:07.952929 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17841,"thread_start_us":17561,"threads_started":1}
I20260807 18:09:07.956056 22274 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46373
I20260807 18:09:07.956421 22274 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:07.957401 22274 heartbeater.cc:511] Master 127.8.62.190:46373 requested a full tablet report, sending...
I20260807 18:09:07.959263 21787 ts_manager.cc:194] Registered new tserver with Master: dcc71732eab04b619e585a1aa9ad9c7f (127.8.62.131:40917)
I20260807 18:09:07.960465 21787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:44601
I20260807 18:09:07.967262 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:09:07.990468 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:07.971303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.8.62.190:46373)
0807 18:09:07.971495 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:07.971501 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:07.972030 (+ 529us) server_negotiation.cc:407] Connection header received
0807 18:09:07.972329 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:07.972335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:07.972436 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:07.972530 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:07.974947 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.976066 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.978138 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:07.978795 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:07.978946 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:07.982740 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:07.982749 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:07.982754 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:07.982806 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:07.986682 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.987729 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.987735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.987739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.987911 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:07.988460 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:07.988467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:07.988471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:07.988911 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:07.989170 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:07.989816 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 18:09:07.990089 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:09:07.998234 21787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:09:08.010404 21787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 07 18:09:08 dist-test-slave-rnrw krb5kdc[20748](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126147, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for HTTP/127.8.62.148@KRBTEST.COM
I20260807 18:09:08.075883 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.050826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.8.62.131:40917)
0807 18:09:08.067194 (+ 16368us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.067202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.067238 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:09:08.067341 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.067353 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.067576 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.067779 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:08.068563 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070223 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.072901 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.074510 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.074640 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.074903 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.075321 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.075513 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16203,"thread_start_us":16040,"threads_started":1}
I20260807 18:09:08.076001 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.050395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:40917 (local address 127.0.0.1:42916)
0807 18:09:08.051031 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:08.051055 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:08.051210 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:08.067773 (+ 16563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:08.067800 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:08.067831 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:08.068222 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.068234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070393 (+ 2159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.070401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.072703 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.072722 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.074775 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.074782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.075021 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.075084 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 18:09:08.075248 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:09:08.075488 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":123,"thread_start_us":2259,"threads_started":1}
I20260807 18:09:08.077925 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.051428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41649 (local address 127.0.0.1:51922)
0807 18:09:08.052099 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:08.052128 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:08.052264 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:08.066495 (+ 14231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:08.066513 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:08.066555 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:08.066971 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.066985 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070647 (+ 3662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.070653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.072703 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.072722 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.077064 (+ 4342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.077070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.077244 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.077277 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 18:09:08.077347 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 18:09:08.077502 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":70,"thread_start_us":819,"threads_started":1}
I20260807 18:09:08.079298 22290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.045009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46601 (local address 127.0.0.1:34870)
0807 18:09:08.048499 (+ 3490us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:08.048568 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:08.048762 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:08.068234 (+ 19472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:08.068239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:08.068270 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:08.068695 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.068709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070684 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.070690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.074388 (+ 3698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.074408 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.078619 (+ 4211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.078627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.078793 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.078827 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:09:08.078904 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:09:08.079070 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3215,"thread_start_us":3047,"threads_started":1}
I20260807 18:09:08.079445 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.052068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.8.62.129:41649)
0807 18:09:08.065542 (+ 13474us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.065555 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.065596 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:09:08.065766 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.065779 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.066157 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.066435 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:08.067287 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070468 (+ 3181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.075033 (+ 4565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.076806 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.078567 (+ 1761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.078838 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.078925 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.079129 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13220,"thread_start_us":13018,"threads_started":1}
I20260807 18:09:08.080272 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.045268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34870 (local address 127.8.62.130:46601)
0807 18:09:08.063211 (+ 17943us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.063225 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.063270 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:09:08.063419 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.063431 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.063811 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.064112 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:08.068935 (+ 4823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.070499 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.074666 (+ 4167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.076700 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.079681 (+ 2981us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.079829 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.079890 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.080026 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17661,"thread_start_us":19102,"threads_started":1}
I20260807 18:09:08.081207 22207 tablet_service.cc:1467] Processing CreateTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 (DEFAULT_TABLE table=test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:08.082675 22207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe25eab0b30a4afd9038ffdc5e60edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:08.083537 21923 tablet_service.cc:1467] Processing CreateTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 (DEFAULT_TABLE table=test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:08.085253 21923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe25eab0b30a4afd9038ffdc5e60edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:08.089880 22070 tablet_service.cc:1467] Processing CreateTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 (DEFAULT_TABLE table=test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:08.091734 22070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe25eab0b30a4afd9038ffdc5e60edb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:08.109659 22296 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Bootstrap starting.
I20260807 18:09:08.116464 22296 tablet_bootstrap.cc:654] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:08.118503 22296 log.cc:824] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:08.122664 22298 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Bootstrap starting.
I20260807 18:09:08.124154 22297 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Bootstrap starting.
I20260807 18:09:08.128149 22297 tablet_bootstrap.cc:654] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:08.128990 22298 tablet_bootstrap.cc:654] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:08.129722 22297 log.cc:824] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:08.130537 22298 log.cc:824] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:08.131116 22296 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: No bootstrap required, opened a new log
I20260807 18:09:08.131574 22296 ts_tablet_manager.cc:1397] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.005s
I20260807 18:09:08.137514 22298 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: No bootstrap required, opened a new log
I20260807 18:09:08.137972 22298 ts_tablet_manager.cc:1397] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260807 18:09:08.143905 22296 raft_consensus.cc:348] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.144485 22296 raft_consensus.cc:374] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:08.144678 22296 raft_consensus.cc:729] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcc71732eab04b619e585a1aa9ad9c7f, State: Initialized, Role: FOLLOWER
I20260807 18:09:08.145469 22296 consensus_queue.cc:260] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.147455 22297 tablet_bootstrap.cc:492] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: No bootstrap required, opened a new log
I20260807 18:09:08.147861 22297 ts_tablet_manager.cc:1397] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.013s
I20260807 18:09:08.151420 22298 raft_consensus.cc:348] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.152062 22298 raft_consensus.cc:374] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:08.152225 22298 raft_consensus.cc:729] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2defaf947d4ecea526c982d38c8abb, State: Initialized, Role: FOLLOWER
I20260807 18:09:08.153195 22298 consensus_queue.cc:260] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.165024 22297 raft_consensus.cc:348] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.165899 22297 raft_consensus.cc:374] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:08.166148 22297 raft_consensus.cc:729] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b235d05782d42b59e1a92d081a2f6fe, State: Initialized, Role: FOLLOWER
I20260807 18:09:08.167440 22297 consensus_queue.cc:260] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.177045 22297 ts_tablet_manager.cc:1428] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Time spent starting tablet: real 0.029s user 0.015s sys 0.011s
I20260807 18:09:08.177724 21990 heartbeater.cc:503] Master 127.8.62.190:46373 was elected leader, sending a full tablet report...
I20260807 18:09:08.192480 22274 heartbeater.cc:503] Master 127.8.62.190:46373 was elected leader, sending a full tablet report...
I20260807 18:09:08.195147 22137 heartbeater.cc:503] Master 127.8.62.190:46373 was elected leader, sending a full tablet report...
I20260807 18:09:08.195961 22298 ts_tablet_manager.cc:1428] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Time spent starting tablet: real 0.058s user 0.029s sys 0.025s
I20260807 18:09:08.197206 22296 ts_tablet_manager.cc:1428] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Time spent starting tablet: real 0.065s user 0.032s sys 0.025s
W20260807 18:09:08.283088 21991 tablet.cc:2369] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:08.397426 22302 raft_consensus.cc:482] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:09:08.397791 22302 raft_consensus.cc:504] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.399858 22302 leader_election.cc:288] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad2defaf947d4ecea526c982d38c8abb (127.8.62.130:46601), 1b235d05782d42b59e1a92d081a2f6fe (127.8.62.129:41649)
I20260807 18:09:08.407230 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.400148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:46601 (local address 127.8.62.131:40823)
0807 18:09:08.400342 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:08.400365 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:08.400518 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:08.401354 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:08.401359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:08.401393 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:08.401660 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.401672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.403021 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.403027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.405146 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.405163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.406438 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.406444 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.406637 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.406663 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:09:08.406776 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:09:08.406925 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:09:08.407219 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.400624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:40823 (local address 127.8.62.130:46601)
0807 18:09:08.400828 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.400834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.400856 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:09:08.400934 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.400938 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.401057 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.401169 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:08.401842 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.402840 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.405347 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.406233 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.406436 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.406516 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.406808 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.406944 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:09:08.407949 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.400771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:58289 (local address 127.8.62.129:41649)
0807 18:09:08.400964 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.400970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.401541 (+ 571us) server_negotiation.cc:407] Connection header received
0807 18:09:08.401799 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.401804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.401903 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.402013 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:08.402610 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.403519 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.405459 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.406693 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.406877 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.406967 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.407548 (+ 581us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.407700 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:09:08.408735 22313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.400206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:41649 (local address 127.8.62.131:58289)
0807 18:09:08.401394 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:08.401409 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:08.401652 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:08.402043 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:08.402048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:08.402075 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:08.402419 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.402430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.403685 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.403698 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.405194 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:08.405208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.407124 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.407130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:08.407322 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.407356 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 18:09:08.408163 (+ 807us) client_negotiation.cc:240] Negotiation successful
0807 18:09:08.408315 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":2,"thread_start_us":889,"threads_started":1}
I20260807 18:09:08.408843 22092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe25eab0b30a4afd9038ffdc5e60edb4" candidate_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2defaf947d4ecea526c982d38c8abb" is_pre_election: true
I20260807 18:09:08.409554 22092 raft_consensus.cc:2454] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc71732eab04b619e585a1aa9ad9c7f in term 0.
I20260807 18:09:08.409924 21944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe25eab0b30a4afd9038ffdc5e60edb4" candidate_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b235d05782d42b59e1a92d081a2f6fe" is_pre_election: true
I20260807 18:09:08.410374 22159 leader_election.cc:302] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2defaf947d4ecea526c982d38c8abb, dcc71732eab04b619e585a1aa9ad9c7f; no voters:
I20260807 18:09:08.410535 21944 raft_consensus.cc:2454] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcc71732eab04b619e585a1aa9ad9c7f in term 0.
I20260807 18:09:08.410866 22302 raft_consensus.cc:2781] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:09:08.411063 22302 raft_consensus.cc:482] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:09:08.411172 22302 raft_consensus.cc:3037] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:08.413962 22302 raft_consensus.cc:504] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.414768 22302 leader_election.cc:288] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [CANDIDATE]: Term 1 election: Requested vote from peers ad2defaf947d4ecea526c982d38c8abb (127.8.62.130:46601), 1b235d05782d42b59e1a92d081a2f6fe (127.8.62.129:41649)
I20260807 18:09:08.415307 22092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe25eab0b30a4afd9038ffdc5e60edb4" candidate_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad2defaf947d4ecea526c982d38c8abb"
I20260807 18:09:08.415562 22092 raft_consensus.cc:3037] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:08.415820 21944 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe25eab0b30a4afd9038ffdc5e60edb4" candidate_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b235d05782d42b59e1a92d081a2f6fe"
I20260807 18:09:08.416059 21944 raft_consensus.cc:3037] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:08.418149 22092 raft_consensus.cc:2454] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcc71732eab04b619e585a1aa9ad9c7f in term 1.
W20260807 18:09:08.418228 22275 tablet.cc:2369] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:08.418987 22159 leader_election.cc:302] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad2defaf947d4ecea526c982d38c8abb, dcc71732eab04b619e585a1aa9ad9c7f; no voters:
I20260807 18:09:08.419426 21944 raft_consensus.cc:2454] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcc71732eab04b619e585a1aa9ad9c7f in term 1.
I20260807 18:09:08.419526 22302 raft_consensus.cc:2781] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:08.420306 22302 raft_consensus.cc:686] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 1 LEADER]: Becoming Leader. State: Replica: dcc71732eab04b619e585a1aa9ad9c7f, State: Running, Role: LEADER
I20260807 18:09:08.420894 22302 consensus_queue.cc:237] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } }
I20260807 18:09:08.425843 21786 catalog_manager.cc:5495] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f reported cstate change: term changed from 0 to 1, leader changed from <none> to dcc71732eab04b619e585a1aa9ad9c7f (127.8.62.131). New cstate: current_term: 1 leader_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcc71732eab04b619e585a1aa9ad9c7f" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 40917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 } health_report { overall_health: UNKNOWN } } }
W20260807 18:09:08.444905 22138 tablet.cc:2369] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:08.521770 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.515548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.8.62.131:40917)
0807 18:09:08.515807 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.515813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.516167 (+ 354us) server_negotiation.cc:407] Connection header received
0807 18:09:08.516423 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.516428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.516511 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.516748 (+ 237us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:09:08.517176 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.518003 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.519478 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.519901 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.520042 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.520637 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:09:08.521161 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:09:08.521326 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.521395 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.521479 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:09:08.530894 21944 raft_consensus.cc:1261] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 1 FOLLOWER]: Refusing update from remote peer dcc71732eab04b619e585a1aa9ad9c7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:08.530992 22092 raft_consensus.cc:1261] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 1 FOLLOWER]: Refusing update from remote peer dcc71732eab04b619e585a1aa9ad9c7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:08.532102 22314 consensus_queue.cc:1035] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b235d05782d42b59e1a92d081a2f6fe" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 41649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:08.532958 22302 consensus_queue.cc:1035] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad2defaf947d4ecea526c982d38c8abb" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 46601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:08.551611 22318 mvcc.cc:205] Tried to move back new op lower bound from 7315972704371408896 to 7315972703936315392. Current Snapshot: MvccSnapshot[applied={T|T < 7315972704371408896}]
I20260807 18:09:08.572064 21786 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:08.572358 21786 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:08.576164 21786 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e486b80cea7e45b5a5260fe24a02c71f: Sending DeleteTablet for 3 replicas of tablet fe25eab0b30a4afd9038ffdc5e60edb4
I20260807 18:09:08.577623 22207 tablet_service.cc:1514] Processing DeleteTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:08 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42916
I20260807 18:09:08.578246 21923 tablet_service.cc:1514] Processing DeleteTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:08 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:51922
I20260807 18:09:08.578442 22070 tablet_service.cc:1514] Processing DeleteTablet for tablet fe25eab0b30a4afd9038ffdc5e60edb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:08 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34870
I20260807 18:09:08.578609 22327 tablet_replica.cc:330] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: stopping tablet replica
I20260807 18:09:08.579087 22327 raft_consensus.cc:2229] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:09:08.579816 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21825
I20260807 18:09:08.580104 22327 raft_consensus.cc:2258] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:08.582723 22327 ts_tablet_manager.cc:1905] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:08.593154 22329 tablet_replica.cc:330] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: stopping tablet replica
I20260807 18:09:08.596889 22327 ts_tablet_manager.cc:1918] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:08.597388 22327 log.cc:1196] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/fe25eab0b30a4afd9038ffdc5e60edb4
I20260807 18:09:08.598259 22327 ts_tablet_manager.cc:1939] T fe25eab0b30a4afd9038ffdc5e60edb4 P dcc71732eab04b619e585a1aa9ad9c7f: Deleting consensus metadata
I20260807 18:09:08.601171 21771 catalog_manager.cc:4841] TS dcc71732eab04b619e585a1aa9ad9c7f (127.8.62.131:40917): tablet fe25eab0b30a4afd9038ffdc5e60edb4 (table test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]) successfully deleted
I20260807 18:09:08.601806 22329 raft_consensus.cc:2229] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:08.602648 22329 raft_consensus.cc:2258] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:08.610397 22328 tablet_replica.cc:330] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: stopping tablet replica
I20260807 18:09:08.611451 22329 ts_tablet_manager.cc:1905] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:08.613763 22328 raft_consensus.cc:2229] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:08.614460 22328 raft_consensus.cc:2258] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:08.617115 22328 ts_tablet_manager.cc:1905] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:08.619135 22329 ts_tablet_manager.cc:1918] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:08.619361 22329 log.cc:1196] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/fe25eab0b30a4afd9038ffdc5e60edb4
I20260807 18:09:08.620409 22329 ts_tablet_manager.cc:1939] T fe25eab0b30a4afd9038ffdc5e60edb4 P ad2defaf947d4ecea526c982d38c8abb: Deleting consensus metadata
I20260807 18:09:08.622638 21771 catalog_manager.cc:4841] TS ad2defaf947d4ecea526c982d38c8abb (127.8.62.130:46601): tablet fe25eab0b30a4afd9038ffdc5e60edb4 (table test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]) successfully deleted
I20260807 18:09:08.625571 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.581854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.8.62.129:41649)
0807 18:09:08.582036 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.582042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.591563 (+ 9521us) server_negotiation.cc:407] Connection header received
0807 18:09:08.591714 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.591724 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.591847 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.592000 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:08.595289 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.596833 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.599995 (+ 3162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.601128 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.601277 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.606701 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:08.606732 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:08.606761 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:08.606846 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:08.613173 (+ 6327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.620725 (+ 7552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.620738 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.620744 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.620954 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.623022 (+ 2068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.623030 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.623033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.623643 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:08.623983 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.624821 (+ 838us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.625118 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:09:08.627322 21985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:08.721539 22328 ts_tablet_manager.cc:1918] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:08.721788 22328 log.cc:1196] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/fe25eab0b30a4afd9038ffdc5e60edb4
I20260807 18:09:08.722515 22328 ts_tablet_manager.cc:1939] T fe25eab0b30a4afd9038ffdc5e60edb4 P 1b235d05782d42b59e1a92d081a2f6fe: Deleting consensus metadata
I20260807 18:09:08.724220 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21825
I20260807 18:09:08.724300 21772 catalog_manager.cc:4841] TS 1b235d05782d42b59e1a92d081a2f6fe (127.8.62.129:41649): tablet fe25eab0b30a4afd9038ffdc5e60edb4 (table test-table [id=0ecb74f25a7c46fd903c7451bbb54b34]) successfully deleted
I20260807 18:09:08.751195 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21994
I20260807 18:09:08.767482 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.752711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.8.62.130:46601)
0807 18:09:08.752894 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.752900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.752920 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:09:08.753160 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.753165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.753248 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.753351 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:08.755259 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.756148 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.757531 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.757965 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.758117 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.760998 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:08.761013 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:08.761030 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:08.761070 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:08.764259 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.765252 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.765262 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.765264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.765405 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.765840 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.765844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.765847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.766239 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:08.766554 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.766872 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.767111 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:09:08.768385 22132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:08.865391 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21994
I20260807 18:09:08.889493 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22141
I20260807 18:09:08.907388 22293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:08.890605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.8.62.131:40917)
0807 18:09:08.890837 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:08.890843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:08.890884 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:09:08.890956 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:08.890962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:08.891072 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:08.891200 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:08.892733 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.893776 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.895881 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:08.896343 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:08.896460 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:08.900036 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:08.900057 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:08.900073 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:08.900117 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:08.904012 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.904909 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.904926 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.904929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.905090 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:08.905537 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:08.905541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:08.905543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:08.905924 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:08.906263 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:08.906621 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 18:09:08.906880 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:09:08.908288 22269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:09.027165 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22141
I20260807 18:09:09.048772 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 21753
I20260807 18:09:09.050086 21818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:09.168992 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 21753
I20260807 18:09:09.200843 8442 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 18:09:09.547494 8442 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 18:09:09.547780 8442 mini_ranger.cc:66] Stopping Ranger...
I20260807 18:09:09.894019 8442 mini_ranger.cc:68] Stopped Ranger
2026-08-07 18:09:09.894 UTC [20779] LOG: received smart shutdown request
2026-08-07 18:09:09.897 UTC [20779] LOG: background worker "logical replication launcher" (PID 20788) exited with exit code 1
2026-08-07 18:09:09.898 UTC [20783] LOG: shutting down
2026-08-07 18:09:10.080 UTC [20779] LOG: database system is shut down
2026-08-07T18:09:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (56107 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:09:10 dist-test-slave-rnrw krb5kdc[22343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:09:10 dist-test-slave-rnrw krb5kdc[22343](info): set up 2 sockets
Aug 07 18:09:10 dist-test-slave-rnrw krb5kdc[22343](info): commencing operation
krb5kdc: starting...
W20260807 18:09:11.471776 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:09:11 dist-test-slave-rnrw krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:09:11.738770 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46565
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:12.089489 22357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:12.089811 22357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:12.089991 22357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:12.097564 22357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:12.097689 22357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:12.097798 22357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:12.097914 22357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:12.097996 22357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:12.108394 22357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:12.109942 22357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:12.111377 22357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:12.123795 22365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:12.123772 22362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:12.123903 22363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:12.126607 22357 server_base.cc:1034] running on GCE node
I20260807 18:09:12.127804 22357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:12.129560 22357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:12.130885 22357 hybrid_clock.cc:648] HybridClock initialized: now 1786126152130827 us; error 54 us; skew 500 ppm
I20260807 18:09:12.131312 22357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:12.138625 22357 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:12.140470 22357 webserver.cc:460] Webserver started at http://127.8.62.190:40561/ using document root <none> and password file <none>
I20260807 18:09:12.141351 22357 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:12.141513 22357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:12.141863 22357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:12.145323 22357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "b9bc7f183b464584803fe678dfc225f4"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "cdd3dbee35a23e6ba26652d066c60700"
server_key_iv: "caf0c6a5c63d240059bdf71bb0e7b50c"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.146214 22357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "b9bc7f183b464584803fe678dfc225f4"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "cdd3dbee35a23e6ba26652d066c60700"
server_key_iv: "caf0c6a5c63d240059bdf71bb0e7b50c"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.152154 22357 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:09:12.156446 22372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:12.158274 22357 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:12.158502 22357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "b9bc7f183b464584803fe678dfc225f4"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "cdd3dbee35a23e6ba26652d066c60700"
server_key_iv: "caf0c6a5c63d240059bdf71bb0e7b50c"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.158757 22357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:12.180495 22357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:12.185990 22357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:12.186437 22357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:12.207690 22357 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46565
I20260807 18:09:12.207753 22424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46565 every 8 connection(s)
I20260807 18:09:12.209308 22357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:12.213460 22425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:12.215653 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22357
I20260807 18:09:12.215898 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:09:12.216845 8442 external_mini_cluster.cc:1428] Setting key e7f9f1c41f881441884c78fa4cec2d2a
I20260807 18:09:12.221307 22425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.189@KRBTEST.COM" created.
I20260807 18:09:12.249384 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.224987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46565 (local address 127.0.0.1:50612)
0807 18:09:12.226075 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.226137 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:12.228076 (+ 1939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:12.229207 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:12.229214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:12.229926 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:12.230348 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.230368 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.232938 (+ 2570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.232944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.234134 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.234148 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.235676 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.235683 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.235843 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.236727 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:12.236753 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:12.240196 (+ 3443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:12.244492 (+ 4296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.244512 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.244531 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.245221 (+ 690us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.245788 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.245792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.245794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.245964 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.247170 (+ 1206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:12.247177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:12.247380 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 18:09:12.247672 (+ 292us) client_negotiation.cc:240] Negotiation successful
0807 18:09:12.248123 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":528,"threads_started":1}
I20260807 18:09:12.249492 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.224983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50612 (local address 127.8.62.190:46565)
0807 18:09:12.226169 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:12.226180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:12.226240 (+ 60us) server_negotiation.cc:407] Connection header received
0807 18:09:12.228524 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:12.228535 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:12.228927 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:12.229177 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:12.230589 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.232745 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.234302 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.235388 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.235604 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.240647 (+ 5043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:12.240661 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:12.240677 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:12.240723 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:12.244251 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.245383 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.245396 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.245402 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.245654 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.246098 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.246106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.246113 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.246988 (+ 875us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:12.247220 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:12.247865 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 18:09:12.248322 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":940,"spinlock_wait_cycles":19328,"thread_start_us":727,"threads_started":1}
I20260807 18:09:12.252985 22425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:09:12.254400 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.253856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38778)
0807 18:09:12.254072 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.254226 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:09:12.254806 22376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
I20260807 18:09:12.256702 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.256158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38782)
0807 18:09:12.256367 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.256474 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:09:12.257088 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:41769: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
I20260807 18:09:12.278801 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:41769
--webserver_interface=127.8.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:12.306490 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } attempt: 1
I20260807 18:09:12.307914 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.307300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38798)
0807 18:09:12.307595 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.307718 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:09:12.309243 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.308748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38810)
0807 18:09:12.308924 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.309023 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 18:09:12.309465 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:41769: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
I20260807 18:09:12.377790 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } attempt: 2
I20260807 18:09:12.379158 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.378585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38822)
0807 18:09:12.378840 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.378960 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 18:09:12.380354 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.379890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38826)
0807 18:09:12.380075 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.380185 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:09:12.380661 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:41769: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
I20260807 18:09:12.471954 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } attempt: 3
I20260807 18:09:12.473330 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.472743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38840)
0807 18:09:12.472988 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.473125 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:09:12.474695 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.474219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38846)
0807 18:09:12.474392 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.474508 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:09:12.474972 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:41769: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
I20260807 18:09:12.618307 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } attempt: 4
I20260807 18:09:12.619801 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.619040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38854)
0807 18:09:12.619409 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.619530 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
I20260807 18:09:12.621145 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.620683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38870)
0807 18:09:12.620851 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.620976 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:09:12.621371 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:41769: Network error: Client connection negotiation failed: client connection to 127.8.62.189:41769: connect: Connection refused (error 111)
W20260807 18:09:12.636448 22432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:12.636948 22432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:12.637146 22432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:12.644942 22432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:12.645078 22432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:12.645169 22432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:12.645275 22432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:12.645360 22432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:12.656319 22432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:41769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:12.657851 22432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:12.659210 22432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:12.671947 22437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:12.672039 22438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:12.672433 22440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:12.672901 22432 server_base.cc:1034] running on GCE node
I20260807 18:09:12.673774 22432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:12.675263 22432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:12.676476 22432 hybrid_clock.cc:648] HybridClock initialized: now 1786126152676441 us; error 28 us; skew 500 ppm
I20260807 18:09:12.676882 22432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:12.680241 22432 init.cc:377] Logged in from keytab as kudu/127.8.62.189@KRBTEST.COM (short username kudu)
I20260807 18:09:12.681954 22432 webserver.cc:460] Webserver started at http://127.8.62.189:43503/ using document root <none> and password file <none>
I20260807 18:09:12.682670 22432 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:12.682786 22432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:12.683043 22432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:12.686355 22432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data/instance:
uuid: "6282bd168c4b4b5da413142b43b31b21"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "e1a9ca25b888354afca48b078ae3f2ac"
server_key_iv: "3a15fde3b2c5bfd93f563a449fbaa3da"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.687211 22432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal/instance:
uuid: "6282bd168c4b4b5da413142b43b31b21"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "e1a9ca25b888354afca48b078ae3f2ac"
server_key_iv: "3a15fde3b2c5bfd93f563a449fbaa3da"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.693102 22432 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260807 18:09:12.697139 22448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:12.698742 22432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:09:12.698912 22432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "6282bd168c4b4b5da413142b43b31b21"
format_stamp: "Formatted at 2026-08-07 18:09:12 on dist-test-slave-rnrw"
server_key: "e1a9ca25b888354afca48b078ae3f2ac"
server_key_iv: "3a15fde3b2c5bfd93f563a449fbaa3da"
server_key_version: "encryptionkey@0"
I20260807 18:09:12.699111 22432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:12.713979 22432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:12.719014 22432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:12.719280 22432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:12.739547 22432 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.189:41769
I20260807 18:09:12.739652 22500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.189:41769 every 8 connection(s)
I20260807 18:09:12.741291 22432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
I20260807 18:09:12.745236 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22432
I20260807 18:09:12.745471 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-1/wal/instance
I20260807 18:09:12.745939 22501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:12.746052 8442 external_mini_cluster.cc:1428] Setting key cb83e00f92a21f60d68ea12da0c9d886
I20260807 18:09:12.757213 22501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.188@KRBTEST.COM" created.
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:12.785970 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.760685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46565 (local address 127.0.0.1:50624)
0807 18:09:12.761636 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.761692 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:12.763876 (+ 2184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:12.764940 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:12.764951 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:12.765563 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:12.766549 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.766572 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.768524 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.768530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.770012 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.770028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.771156 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.771167 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.771423 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.772194 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:12.772220 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:12.774990 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:12.779853 (+ 4863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.779868 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.779893 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.780769 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.781407 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.781413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.781416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.781739 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.782771 (+ 1032us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:12.782780 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:12.783336 (+ 556us) client_negotiation.cc:769] Sending connection context
0807 18:09:12.783706 (+ 370us) client_negotiation.cc:240] Negotiation successful
0807 18:09:12.784312 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":397,"threads_started":1}
I20260807 18:09:12.786060 22505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.760165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.8.62.190:46565)
0807 18:09:12.760828 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:12.760835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:12.761838 (+ 1003us) server_negotiation.cc:407] Connection header received
0807 18:09:12.764222 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:12.764232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:12.764344 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:12.764489 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:12.766828 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.768291 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.770199 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.770879 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.770996 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.775297 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:12.775303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:12.775306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:12.775360 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:12.779579 (+ 4219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.780967 (+ 1388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.780984 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.780997 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.781185 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.781928 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.781936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.781942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.782552 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:12.782761 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:12.784897 (+ 2136us) server_negotiation.cc:299] Negotiation successful
0807 18:09:12.785332 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":379,"threads_started":1}
I20260807 18:09:12.788738 22501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
I20260807 18:09:12.809140 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:42957
--webserver_interface=127.8.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:12.812086 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.789960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38872 (local address 127.8.62.189:41769)
0807 18:09:12.790780 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:12.790787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:12.790819 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:09:12.790876 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:12.790882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:12.791094 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:12.791238 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:12.792309 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.793494 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.795488 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.796274 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.796383 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.804424 (+ 8041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:12.804484 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:12.804500 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:12.804626 (+ 126us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:12.808266 (+ 3640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.809368 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.809382 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.809387 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.809536 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.810137 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.810141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.810143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.810578 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:12.810764 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:12.811346 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:09:12.811711 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":645,"thread_start_us":487,"threads_started":1}
I20260807 18:09:12.812335 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.789645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38872)
0807 18:09:12.789999 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.790022 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:12.790143 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:12.791289 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:12.791301 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:12.791875 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:12.792106 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.792123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.793665 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.793671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.795308 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.795329 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.796473 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.796486 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.796781 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.801135 (+ 4354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:12.801159 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:12.804148 (+ 2989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:12.808519 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.808529 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.808536 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.809180 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.809712 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.809720 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.809728 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.809972 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.810762 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:12.810770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:12.810970 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 18:09:12.811163 (+ 193us) client_negotiation.cc:240] Negotiation successful
0807 18:09:12.811647 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196}
I20260807 18:09:12.815493 22501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:09:12.816855 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.816274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56782)
0807 18:09:12.816479 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.816690 (+ 211us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 18:09:12.817170 22449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:12.819152 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.818615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56784)
0807 18:09:12.818827 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.818961 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 18:09:12.819454 22501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:12.868857 22501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 1
I20260807 18:09:12.870308 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.869719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56790)
0807 18:09:12.869977 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.870114 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:09:12.871423 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.871020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56794)
0807 18:09:12.871186 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.871287 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 18:09:12.871665 22501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:12.920693 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } attempt: 5
Aug 07 18:09:12 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:09:12.936952 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.921473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38876)
0807 18:09:12.921736 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.921760 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:12.921905 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:12.922439 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:12.922444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:12.922861 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:12.923069 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.923078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.924500 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.924505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.925460 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:12.925475 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.926523 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.926529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:12.926670 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.927358 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:12.927377 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:12.929851 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:12.934174 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.934180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.934184 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.934642 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.935114 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:12.935119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:12.935121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:12.935266 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:12.935946 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:12.935952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:12.936120 (+ 168us) client_negotiation.cc:769] Sending connection context
0807 18:09:12.936285 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 18:09:12.936685 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:09:12.937054 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.921830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.8.62.189:41769)
0807 18:09:12.922063 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:12.922070 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:12.922092 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:09:12.922159 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:12.922164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:12.922257 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:12.922374 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:12.923246 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.924341 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.925645 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:12.926327 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:12.926477 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:12.930151 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:12.930161 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:12.930168 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:12.930222 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:12.933933 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.934786 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.934792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.934795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.934967 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:12.935388 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:12.935393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:12.935396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:12.935784 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:12.935944 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:12.936424 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 18:09:12.936717 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 18:09:12.938261 22425 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:09:12.939502 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.938958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56806)
0807 18:09:12.939192 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.939292 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 18:09:12.939901 22501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 2
I20260807 18:09:12.940692 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.940249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56812)
0807 18:09:12.940412 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.940506 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:09:12.941130 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.940487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56814)
0807 18:09:12.940823 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.940963 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
W20260807 18:09:12.941408 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:12.942214 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.941871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56830)
0807 18:09:12.942015 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.942100 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:09:12.942445 22501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:12.992653 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 1
I20260807 18:09:12.993952 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.993457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56846)
0807 18:09:12.993661 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.993770 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:09:12.995203 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:12.994760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56860)
0807 18:09:12.994921 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:12.995034 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:09:12.995458 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:13.033663 22501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 3
I20260807 18:09:13.035074 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.034559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56872)
0807 18:09:13.034814 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.034912 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 18:09:13.036429 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.035999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56884)
0807 18:09:13.036246 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.036324 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
W20260807 18:09:13.036780 22501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:13.063747 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 2
I20260807 18:09:13.065078 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.064521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56898)
0807 18:09:13.064796 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.064906 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 18:09:13.066239 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.065776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56914)
0807 18:09:13.065992 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.066095 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:09:13.066428 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:13.172725 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 3
I20260807 18:09:13.174145 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.173620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56920)
0807 18:09:13.173843 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.173946 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:09:13.175253 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.174822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56924)
0807 18:09:13.175060 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.175133 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 18:09:13.175465 22425 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
I20260807 18:09:13.180029 22501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 4
I20260807 18:09:13.181315 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.180793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56938)
0807 18:09:13.181049 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.181147 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:09:13.182389 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.182045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56948)
0807 18:09:13.182198 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.182279 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:09:13.182618 22501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:42957: Network error: Client connection negotiation failed: client connection to 127.8.62.188:42957: connect: Connection refused (error 111)
W20260807 18:09:13.185956 22510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:13.186408 22510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:13.186573 22510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:13.194379 22510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:13.194550 22510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:13.194664 22510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:13.194768 22510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:13.194855 22510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:13.205442 22510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46565,127.8.62.189:41769,127.8.62.188:42957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:13.207233 22510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:13.209131 22510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:13.224118 22516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:13.224153 22515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:13.224213 22518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:13.225126 22510 server_base.cc:1034] running on GCE node
I20260807 18:09:13.226008 22510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:13.227567 22510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:13.228814 22510 hybrid_clock.cc:648] HybridClock initialized: now 1786126153228749 us; error 53 us; skew 500 ppm
I20260807 18:09:13.229194 22510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126153, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:13.232525 22510 init.cc:377] Logged in from keytab as kudu/127.8.62.188@KRBTEST.COM (short username kudu)
I20260807 18:09:13.234202 22510 webserver.cc:460] Webserver started at http://127.8.62.188:45735/ using document root <none> and password file <none>
I20260807 18:09:13.234959 22510 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:13.235113 22510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:13.235437 22510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:13.238770 22510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data/instance:
uuid: "e8f16cc956b44962a72624ecc4457212"
format_stamp: "Formatted at 2026-08-07 18:09:13 on dist-test-slave-rnrw"
server_key: "c498eb743e807b51c354cce6a9ae05ea"
server_key_iv: "edc43c4935a96772e85cf695dcacf681"
server_key_version: "encryptionkey@0"
I20260807 18:09:13.239684 22510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal/instance:
uuid: "e8f16cc956b44962a72624ecc4457212"
format_stamp: "Formatted at 2026-08-07 18:09:13 on dist-test-slave-rnrw"
server_key: "c498eb743e807b51c354cce6a9ae05ea"
server_key_iv: "edc43c4935a96772e85cf695dcacf681"
server_key_version: "encryptionkey@0"
I20260807 18:09:13.245676 22510 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 18:09:13.249776 22526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:13.251379 22510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:09:13.251684 22510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "e8f16cc956b44962a72624ecc4457212"
format_stamp: "Formatted at 2026-08-07 18:09:13 on dist-test-slave-rnrw"
server_key: "c498eb743e807b51c354cce6a9ae05ea"
server_key_iv: "edc43c4935a96772e85cf695dcacf681"
server_key_version: "encryptionkey@0"
I20260807 18:09:13.251919 22510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:13.280020 22510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:13.284873 22510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:13.285197 22510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:13.305949 22510 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.188:42957
I20260807 18:09:13.305981 22578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.188:42957 every 8 connection(s)
I20260807 18:09:13.307555 22510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
I20260807 18:09:13.309015 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22510
I20260807 18:09:13.309216 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786125932226923-8442-0/minicluster-data/master-2/wal/instance
I20260807 18:09:13.309913 8442 external_mini_cluster.cc:1428] Setting key eeb2c15e14aa517be97ee6cc83842fc0
I20260807 18:09:13.312687 22579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:13.321669 22579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126153, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:13.336066 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.312073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.8.62.188:42957)
0807 18:09:13.312996 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.313010 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.313060 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:09:13.314755 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.314780 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.315209 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.315612 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.317691 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.319198 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.321275 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.322091 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.322267 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.327668 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.327677 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.327689 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.327741 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.331036 (+ 3295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.331884 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.331892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.331899 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.332081 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.332594 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.332602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.332605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.333249 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.333579 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.334100 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.334565 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":440,"threads_started":1}
I20260807 18:09:13.341746 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.325275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.8.62.190:46565)
0807 18:09:13.325874 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.325881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.326596 (+ 715us) server_negotiation.cc:407] Connection header received
0807 18:09:13.326820 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.326828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.326940 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.327082 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.328423 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.329258 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.330770 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.331206 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.331383 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.334983 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.334992 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.334997 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.335043 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.338333 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.339263 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.339271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.339275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.339461 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.339989 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.339997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.340004 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.340354 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.340527 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.341045 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.341297 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":289,"threads_started":1}
I20260807 18:09:13.341759 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.325275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46565 (local address 127.0.0.1:50628)
0807 18:09:13.326422 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.326478 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:13.326664 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:13.327142 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:13.327147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:13.327873 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:13.328238 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.328248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.329428 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.329440 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.330591 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.330610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.331413 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.331420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.331614 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.332266 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:13.332285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:13.334738 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:13.338486 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.338498 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.338505 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.339116 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.339619 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.339623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.339646 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.339836 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.340497 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:13.340505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:13.340732 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 18:09:13.340929 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 18:09:13.341297 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":574,"threads_started":1}
I20260807 18:09:13.343173 22579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126153, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:09:13.352654 22425 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 4
I20260807 18:09:13.358625 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.343854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:41769 (local address 127.0.0.1:38884)
0807 18:09:13.344058 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.344079 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:13.344192 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:13.344489 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:13.344495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:13.345010 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:13.345209 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.345221 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.346247 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.346253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.347161 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.347174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.347922 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.347928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.348140 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.348754 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:13.348771 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:13.351120 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:13.355156 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.355164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.355169 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.355775 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.356236 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.356241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.356244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.356468 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.357198 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:13.357203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:13.357718 (+ 515us) client_negotiation.cc:769] Sending connection context
0807 18:09:13.357895 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 18:09:13.358241 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 18:09:13.358685 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.339668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50644 (local address 127.8.62.190:46565)
0807 18:09:13.340352 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.340356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.340376 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:09:13.340418 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.340421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.340475 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.340645 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.342567 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.343488 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.345549 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.346011 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.346158 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.351034 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.351040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.351043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.351089 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.355244 (+ 4155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.356150 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.356157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.356164 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.356327 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.356863 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.356868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.356871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.357297 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.357639 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.357865 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.358116 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":325,"threads_started":1}
I20260807 18:09:13.359747 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.344026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.8.62.189:41769)
0807 18:09:13.344206 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.344211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.344228 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:09:13.344281 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.344285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.344342 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.344421 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.345379 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.346096 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.347317 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.347756 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.347888 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.351330 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.351336 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.351341 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.351381 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.355011 (+ 3630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.355918 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.355924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.355927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.356125 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.356616 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.356621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.356624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.357032 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.357242 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.359078 (+ 1836us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.359337 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:09:13.361290 22579 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:09:13.371331 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.353454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56972 (local address 127.8.62.188:42957)
0807 18:09:13.353638 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.353643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.353741 (+ 98us) server_negotiation.cc:407] Connection header received
0807 18:09:13.353912 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.353918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.354003 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.354128 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.355252 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.356346 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.358164 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.358854 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.358977 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.363250 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.363259 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.363264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.363310 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.367931 (+ 4621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.368885 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.368891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.368897 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.369117 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.369618 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.369626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.369629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.370078 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.370269 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.370896 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.371099 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:09:13.371502 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.353406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56972)
0807 18:09:13.353633 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.353655 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:13.353784 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:13.354177 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:13.354182 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:13.354841 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:13.355086 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.355101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.356597 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.356604 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.357996 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.358012 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.359175 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.359181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.359385 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.360096 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:13.360120 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:13.363010 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:13.368111 (+ 5101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.368118 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.368122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.368747 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.369271 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.369277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.369280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.369471 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.370343 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:13.370349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:13.370563 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 18:09:13.370736 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 18:09:13.371088 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 18:09:13.378655 22425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: Bootstrap starting.
I20260807 18:09:13.379500 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.361208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38892 (local address 127.8.62.189:41769)
0807 18:09:13.361367 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.361372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.361388 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:09:13.361435 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.361439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.361529 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.361624 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.363640 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.364795 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.366586 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.367112 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.367256 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.372689 (+ 5433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.372694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.372698 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.372733 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.376328 (+ 3595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.377233 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.377239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.377242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.377431 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.377896 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.377901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.377904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.378307 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.378626 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.378891 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.379157 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126153, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:09:13.383735 22425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:13.385331 22425 log.cc:824] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:13.385476 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:09:13.388136 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.364465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56974)
0807 18:09:13.364993 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.365013 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:13.365135 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:13.365485 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:13.365489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:13.365919 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:13.366046 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.366053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.367843 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.367849 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.369259 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.369274 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.370164 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.370170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.370374 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.371648 (+ 1274us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:13.371669 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:13.380451 (+ 8782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:13.385025 (+ 4574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.385031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.385035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.385598 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.386078 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.386083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.386088 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.386254 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.387038 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:13.387042 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:13.387164 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 18:09:13.387327 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 18:09:13.387719 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291}
I20260807 18:09:13.388281 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.363169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56974 (local address 127.8.62.188:42957)
0807 18:09:13.365184 (+ 2015us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.365193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.365217 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:09:13.365267 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.365273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.365362 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.365470 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.366239 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.367689 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.369420 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.369983 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.370142 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.380897 (+ 10755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.380902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.380905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.380940 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.384762 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.385731 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.385736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.385741 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.385913 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.386386 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.386393 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.386398 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.386831 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.386986 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.387456 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.387860 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":185,"server-negotiator.queue_time_us":1855,"thread_start_us":681,"threads_started":1}
I20260807 18:09:13.390110 22425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: No bootstrap required, opened a new log
I20260807 18:09:13.393461 22579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: Bootstrap starting.
I20260807 18:09:13.396886 22579 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:13.398147 22579 log.cc:824] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:13.405089 22425 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.405598 22425 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:13.405745 22425 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9bc7f183b464584803fe678dfc225f4, State: Initialized, Role: FOLLOWER
I20260807 18:09:13.406634 22425 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.410244 22596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.410720 22596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.411051 22579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: No bootstrap required, opened a new log
I20260807 18:09:13.412716 22425 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:13.424399 22579 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.424988 22579 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:13.425140 22579 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f16cc956b44962a72624ecc4457212, State: Initialized, Role: FOLLOWER
I20260807 18:09:13.426083 22579 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.427258 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.399574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.8.62.190:46565)
0807 18:09:13.399823 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.399832 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.403023 (+ 3191us) server_negotiation.cc:407] Connection header received
0807 18:09:13.403251 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.403259 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.403375 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.403485 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.410172 (+ 6687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.411317 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.414560 (+ 3243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.415272 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.415385 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.419305 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.419313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.419318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.419367 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.423090 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.423979 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.423985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.423989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.424179 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.424749 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.424754 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.424757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.425413 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.425680 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.426307 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.426706 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:09:13.432145 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.398996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38898 (local address 127.8.62.189:41769)
0807 18:09:13.399191 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.399199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.404101 (+ 4902us) server_negotiation.cc:407] Connection header received
0807 18:09:13.407015 (+ 2914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.407023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.407123 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.407214 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.413449 (+ 6235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.414581 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.418059 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.418990 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.419099 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.423113 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.423124 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.423132 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.423203 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.427450 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.428796 (+ 1346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.428803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.428806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.429007 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.429576 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.429583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.429586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.430033 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.430306 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.431427 (+ 1121us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.431729 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:09:13.434607 22600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.435178 22600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.436625 22579 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:13.438191 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.403255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56986 (local address 127.8.62.188:42957)
0807 18:09:13.403462 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.403469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.414720 (+ 11251us) server_negotiation.cc:407] Connection header received
0807 18:09:13.415033 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.415039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.415126 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.415206 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.418953 (+ 3747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.420117 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.425044 (+ 4927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.425903 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.426025 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.430749 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.430756 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.430763 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.430805 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.434738 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.435784 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.435792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.435799 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.435995 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.436531 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.436535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.436537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.436983 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.437275 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.437612 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.437847 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 18:09:13.439642 22378 tablet.cc:2369] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:09:13.445497 22610 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:09:13.445765 22610 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:09:13.446094 22357 master_runner.cc:186] Error getting master registration for 127.8.62.189:41769: OK, instance_id { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" instance_seqno: 1786126152733363 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:09:13.459758 22357 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 18:09:13.467363 22622 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:09:13.467694 22622 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 18:09:13.458300 22510 master_runner.cc:186] Error getting master registration for 127.8.62.189:41769: OK, instance_id { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" instance_seqno: 1786126152733363 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:09:13.469856 22510 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:09:13.481817 22501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } attempt: 5
Aug 07 18:09:13 dist-test-slave-rnrw krb5kdc[22343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126152, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:09:13.502568 22586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.482604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.8.62.188:42957)
0807 18:09:13.482779 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:13.482785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:13.482804 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:09:13.482896 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:13.482902 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:13.482985 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:13.483106 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:13.484154 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.485434 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.487233 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.487943 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.488083 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.494542 (+ 6459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:13.494551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:13.494556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:13.494600 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:13.498904 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.499938 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.499945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.499949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.500162 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:13.500702 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:13.500709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:13.500712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:13.501165 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:13.501348 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:13.501836 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 18:09:13.502096 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:09:13.505057 22506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:13.482410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:42957 (local address 127.0.0.1:56990)
0807 18:09:13.482635 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:13.482657 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:13.482774 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:13.483173 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:13.483179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:13.483741 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:13.483978 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.483992 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.485604 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.485610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.487055 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:13.487073 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:13.488129 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:13.488135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:13.488449 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:13.489263 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:13.489290 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:13.494297 (+ 5007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:13.499118 (+ 4821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.499126 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.499130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.499775 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.500315 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:13.500321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:13.500325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:13.500526 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:13.501308 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:13.501313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:13.501510 (+ 197us) client_negotiation.cc:769] Sending connection context
0807 18:09:13.501693 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 18:09:13.502033 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:09:13.511333 22501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: Bootstrap starting.
I20260807 18:09:13.514376 22596 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:09:13.514727 22596 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.515836 22501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:13.517320 22501 log.cc:824] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:13.517546 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9bc7f183b464584803fe678dfc225f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f16cc956b44962a72624ecc4457212" is_pre_election: true
I20260807 18:09:13.518258 22554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9bc7f183b464584803fe678dfc225f4 in term 0.
I20260807 18:09:13.518999 22373 leader_election.cc:302] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9bc7f183b464584803fe678dfc225f4, e8f16cc956b44962a72624ecc4457212; no voters:
I20260807 18:09:13.516458 22596 leader_election.cc:288] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6282bd168c4b4b5da413142b43b31b21 (127.8.62.189:41769), e8f16cc956b44962a72624ecc4457212 (127.8.62.188:42957)
I20260807 18:09:13.524801 22596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:09:13.525029 22596 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:09:13.525166 22596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:13.528316 22501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: No bootstrap required, opened a new log
I20260807 18:09:13.530712 22596 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
W20260807 18:09:13.537534 22533 tablet.cc:2369] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:13.541179 22501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.541785 22501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:13.541960 22501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6282bd168c4b4b5da413142b43b31b21, State: Initialized, Role: FOLLOWER
I20260807 18:09:13.542857 22501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.536827 22596 leader_election.cc:288] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [CANDIDATE]: Term 1 election: Requested vote from peers 6282bd168c4b4b5da413142b43b31b21 (127.8.62.189:41769), e8f16cc956b44962a72624ecc4457212 (127.8.62.188:42957)
I20260807 18:09:13.550148 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.552434 22554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9bc7f183b464584803fe678dfc225f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8f16cc956b44962a72624ecc4457212"
I20260807 18:09:13.552847 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.552819 22554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:13.552692 22501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:13.552781 22474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9bc7f183b464584803fe678dfc225f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6282bd168c4b4b5da413142b43b31b21"
I20260807 18:09:13.553421 22474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:13.557284 22554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9bc7f183b464584803fe678dfc225f4 in term 1.
I20260807 18:09:13.557401 22474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9bc7f183b464584803fe678dfc225f4 in term 1.
I20260807 18:09:13.557969 22475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9bc7f183b464584803fe678dfc225f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6282bd168c4b4b5da413142b43b31b21" is_pre_election: true
I20260807 18:09:13.558218 22373 leader_election.cc:302] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9bc7f183b464584803fe678dfc225f4, e8f16cc956b44962a72624ecc4457212; no voters:
I20260807 18:09:13.558442 22475 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate b9bc7f183b464584803fe678dfc225f4 in term 1. Re-sending same reply.
I20260807 18:09:13.558844 22626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:13.559377 22626 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [term 1 LEADER]: Becoming Leader. State: Replica: b9bc7f183b464584803fe678dfc225f4, State: Running, Role: LEADER
I20260807 18:09:13.560472 22626 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } }
I20260807 18:09:13.576328 22596 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9bc7f183b464584803fe678dfc225f4. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.576807 22596 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:13.587227 22633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:13.593885 22633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:13.613519 22554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [term 1 FOLLOWER]: Refusing update from remote peer b9bc7f183b464584803fe678dfc225f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:13.615811 22596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:13.625175 22474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [term 1 FOLLOWER]: Refusing update from remote peer b9bc7f183b464584803fe678dfc225f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:13.626704 22596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:13.627728 22641 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:09:13.657277 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9bc7f183b464584803fe678dfc225f4. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.657766 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.664906 22637 mvcc.cc:205] Tried to move back new op lower bound from 7315972725178634240 to 7315972724997218304. Current Snapshot: MvccSnapshot[applied={T|T < 7315972725178634240}]
I20260807 18:09:13.675556 22627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.675973 22627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.677253 22644 mvcc.cc:205] Tried to move back new op lower bound from 7315972725178634240 to 7315972724997218304. Current Snapshot: MvccSnapshot[applied={T|T < 7315972725178634240}]
I20260807 18:09:13.679299 22600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9bc7f183b464584803fe678dfc225f4. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.679708 22600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.687136 22633 catalog_manager.cc:1347] Generated new cluster ID: 885b8639a9ca4b1d972141c79eeac60c
I20260807 18:09:13.687310 22633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:13.687320 22626 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.689488 22626 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:13.689888 22626 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.690548 22626 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:13.703116 22600 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b9bc7f183b464584803fe678dfc225f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9bc7f183b464584803fe678dfc225f4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46565 } } peers { permanent_uuid: "6282bd168c4b4b5da413142b43b31b21" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 41769 } } peers { permanent_uuid: "e8f16cc956b44962a72624ecc4457212" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 42957 } } }
I20260807 18:09:13.703400 22600 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:09:13.725773 22633 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:09:13.727994 22633 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:13.754671 22633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9bc7f183b464584803fe678dfc225f4: Generated new TSK 0
I20260807 18:09:13.756028 22633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:09:14.469888 22622 catalog_manager.cc:1259] Loaded cluster ID: 885b8639a9ca4b1d972141c79eeac60c
I20260807 18:09:14.469993 22622 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: loading cluster ID for follower catalog manager: success
I20260807 18:09:14.472633 22622 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: acquiring CA information for follower catalog manager: success
I20260807 18:09:14.474603 22622 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8f16cc956b44962a72624ecc4457212: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:09:14.632674 22641 catalog_manager.cc:1259] Loaded cluster ID: 885b8639a9ca4b1d972141c79eeac60c
I20260807 18:09:14.632822 22641 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: loading cluster ID for follower catalog manager: success
I20260807 18:09:14.635828 22641 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: acquiring CA information for follower catalog manager: success
I20260807 18:09:14.638358 22641 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6282bd168c4b4b5da413142b43b31b21: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:09:14.868125 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22357
I20260807 18:09:14.869454 22419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:14.985584 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22357
W20260807 18:09:15.011169 22451 connection.cc:535] server connection from 127.0.0.1:38876 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:09:15.011837 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22432
I20260807 18:09:15.013139 22496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:15.123167 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22432
I20260807 18:09:15.145833 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22510
I20260807 18:09:15.147130 22574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:15.258524 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22510
2026-08-07T18:09:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5036 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:09:15 dist-test-slave-rnrw krb5kdc[22660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:09:15 dist-test-slave-rnrw krb5kdc[22660](info): set up 2 sockets
Aug 07 18:09:15 dist-test-slave-rnrw krb5kdc[22660](info): commencing operation
krb5kdc: starting...
W20260807 18:09:16.419914 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:09:16 dist-test-slave-rnrw krb5kdc[22660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:09:16.693238 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44857
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:37465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:44857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:17.062405 22674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:17.062742 22674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:17.062922 22674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:17.070551 22674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:17.070686 22674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:17.070776 22674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:17.070873 22674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:17.070964 22674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:17.081648 22674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:37465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:44857
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:44857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:17.083160 22674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:17.084625 22674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:17.097951 22679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:17.098011 22680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:17.097920 22682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:17.098946 22674 server_base.cc:1034] running on GCE node
I20260807 18:09:17.099855 22674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:17.101616 22674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:17.102900 22674 hybrid_clock.cc:648] HybridClock initialized: now 1786126157102848 us; error 47 us; skew 500 ppm
I20260807 18:09:17.103387 22674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:17 dist-test-slave-rnrw krb5kdc[22660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126157, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:17.107115 22674 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:17.109191 22674 webserver.cc:460] Webserver started at http://127.8.62.190:38965/ using document root <none> and password file <none>
I20260807 18:09:17.110006 22674 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:17.110175 22674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:17.110526 22674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:17.114499 22674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "ad2a59e9225745449cc86270fc3b9ce8"
format_stamp: "Formatted at 2026-08-07 18:09:17 on dist-test-slave-rnrw"
server_key: "d076a97054fc025393453d68a449c244"
server_key_iv: "9a413d97ebd4747196a29723d054ec36"
server_key_version: "encryptionkey@0"
I20260807 18:09:17.115418 22674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "ad2a59e9225745449cc86270fc3b9ce8"
format_stamp: "Formatted at 2026-08-07 18:09:17 on dist-test-slave-rnrw"
server_key: "d076a97054fc025393453d68a449c244"
server_key_iv: "9a413d97ebd4747196a29723d054ec36"
server_key_version: "encryptionkey@0"
I20260807 18:09:17.121910 22674 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:09:17.127285 22689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:17.129149 22674 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 18:09:17.129405 22674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "ad2a59e9225745449cc86270fc3b9ce8"
format_stamp: "Formatted at 2026-08-07 18:09:17 on dist-test-slave-rnrw"
server_key: "d076a97054fc025393453d68a449c244"
server_key_iv: "9a413d97ebd4747196a29723d054ec36"
server_key_version: "encryptionkey@0"
I20260807 18:09:17.129673 22674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:17.149258 22674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:17.154649 22674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:17.155015 22674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:17.177197 22674 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:44857
I20260807 18:09:17.177242 22741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:44857 every 8 connection(s)
I20260807 18:09:17.178903 22674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:17.181627 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22674
I20260807 18:09:17.181896 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:09:17.182823 8442 external_mini_cluster.cc:1428] Setting key fa5c835a7ed62879b96f17428e63e86e
I20260807 18:09:17.184162 22742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:09:17 dist-test-slave-rnrw krb5kdc[22660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:17.198988 22742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8: Bootstrap starting.
I20260807 18:09:17.203177 22742 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:17.204442 22742 log.cc:824] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:17.208689 22742 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8: No bootstrap required, opened a new log
I20260807 18:09:17.209738 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:17.185121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.8.62.190:44857)
0807 18:09:17.186068 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:17.186083 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:17.186131 (+ 48us) server_negotiation.cc:407] Connection header received
0807 18:09:17.187938 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:17.187966 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:17.188457 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:17.189045 (+ 588us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:17.190974 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:17.192418 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:17.193920 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:17.194683 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:17.194955 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:17.200251 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:17.200265 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:17.200281 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:17.200334 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:17.204509 (+ 4175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:17.205878 (+ 1369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:17.205891 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:17.205902 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:17.206174 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:17.206676 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:17.206680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:17.206682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:17.207305 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:17.207565 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:17.208064 (+ 499us) server_negotiation.cc:299] Negotiation successful
0807 18:09:17.208500 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":681,"thread_start_us":420,"threads_started":1}
I20260807 18:09:17.214222 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:09:17.218855 22742 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } }
I20260807 18:09:17.219341 22742 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:17.219484 22742 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad2a59e9225745449cc86270fc3b9ce8, State: Initialized, Role: FOLLOWER
I20260807 18:09:17.220207 22742 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } }
I20260807 18:09:17.220472 22742 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:17.220687 22742 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:17.220906 22742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:17.223300 22742 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } }
I20260807 18:09:17.223922 22742 leader_election.cc:302] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad2a59e9225745449cc86270fc3b9ce8; no voters:
I20260807 18:09:17.224709 22742 leader_election.cc:288] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:09:17.224985 22747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:17.226027 22747 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [term 1 LEADER]: Becoming Leader. State: Replica: ad2a59e9225745449cc86270fc3b9ce8, State: Running, Role: LEADER
I20260807 18:09:17.226666 22747 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } }
I20260807 18:09:17.227198 22742 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:17.229807 22749 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad2a59e9225745449cc86270fc3b9ce8. Latest consensus state: current_term: 1 leader_uuid: "ad2a59e9225745449cc86270fc3b9ce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } } }
I20260807 18:09:17.229866 22748 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad2a59e9225745449cc86270fc3b9ce8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad2a59e9225745449cc86270fc3b9ce8" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 44857 } } }
I20260807 18:09:17.230787 22749 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:17.230888 22748 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:17.234051 22754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:17.239493 22754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:17.249785 22754 catalog_manager.cc:1347] Generated new cluster ID: f80f60160ace4d59aa27e067146693f9
I20260807 18:09:17.249938 22754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:17.258882 22754 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:09:17.260476 22754 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:17.270903 22754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad2a59e9225745449cc86270fc3b9ce8: Generated new TSK 0
I20260807 18:09:17.272015 22754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:09:17.330060 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22674
I20260807 18:09:17.331570 22737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:17.435461 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22674
2026-08-07T18:09:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2173 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:09:17 dist-test-slave-rnrw krb5kdc[22773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:09:17 dist-test-slave-rnrw krb5kdc[22773](info): set up 2 sockets
Aug 07 18:09:17 dist-test-slave-rnrw krb5kdc[22773](info): commencing operation
krb5kdc: starting...
W20260807 18:09:18.575753 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:09:18 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:09:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:09:18.848807 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:19.209110 22787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:19.209475 22787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:19.209580 22787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:19.217314 22787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:19.217448 22787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:19.217495 22787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:19.217557 22787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:19.217597 22787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:19.228179 22787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:19.229590 22787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:19.230933 22787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:19.243626 22792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:19.243587 22795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:19.243741 22793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:19.245414 22787 server_base.cc:1034] running on GCE node
I20260807 18:09:19.246480 22787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:19.248176 22787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:19.249485 22787 hybrid_clock.cc:648] HybridClock initialized: now 1786126159249416 us; error 72 us; skew 500 ppm
I20260807 18:09:19.249888 22787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:19 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126159, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:19.253571 22787 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:19.255455 22787 webserver.cc:460] Webserver started at http://127.8.62.190:41707/ using document root <none> and password file <none>
I20260807 18:09:19.256246 22787 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:19.256414 22787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:19.256903 22787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:19.260236 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "9b7e4e12855c4a018c3a32d3e284b9cb"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "a5017e4e83223f60850c6f51b522980e"
server_key_iv: "298618ffb9db815e15b6c7074ffb17b3"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.261152 22787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "9b7e4e12855c4a018c3a32d3e284b9cb"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "a5017e4e83223f60850c6f51b522980e"
server_key_iv: "298618ffb9db815e15b6c7074ffb17b3"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.266968 22787 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:09:19.271358 22802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:19.273244 22787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:09:19.273473 22787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "9b7e4e12855c4a018c3a32d3e284b9cb"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "a5017e4e83223f60850c6f51b522980e"
server_key_iv: "298618ffb9db815e15b6c7074ffb17b3"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.273717 22787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:19.304630 22787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:19.310572 22787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:19.310949 22787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:19.332186 22787 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46687
I20260807 18:09:19.332226 22855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46687 every 8 connection(s)
I20260807 18:09:19.334378 22787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:19.336978 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22787
I20260807 18:09:19.337239 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:09:19.338164 8442 external_mini_cluster.cc:1428] Setting key 8f2b5464a908154aaf26457b9f08b224
I20260807 18:09:19.339978 22856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:09:19 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:19.355996 22856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap starting.
I20260807 18:09:19.361236 22856 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:19.362658 22856 log.cc:824] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:19.367175 22856 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: No bootstrap required, opened a new log
I20260807 18:09:19.367471 22860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:19.340738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46168 (local address 127.8.62.190:46687)
0807 18:09:19.341661 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:19.341674 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:19.341718 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:09:19.343356 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:19.343381 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:19.344064 (+ 683us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:19.344628 (+ 564us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:19.346601 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:19.348142 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:19.350134 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:19.350937 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:19.351140 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:19.356452 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:19.356466 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:19.356490 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:19.356575 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:19.361270 (+ 4695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:19.362262 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:19.362276 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:19.362287 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:19.362543 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:19.363149 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:19.363158 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:19.363161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:19.363952 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:19.364422 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:19.365268 (+ 846us) server_negotiation.cc:299] Negotiation successful
0807 18:09:19.365961 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":438,"threads_started":1}
I20260807 18:09:19.375845 22856 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:19.376336 22856 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:19.376531 22856 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Initialized, Role: FOLLOWER
I20260807 18:09:19.377426 22856 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:19.377789 22856 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:19.377938 22856 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:19.378149 22856 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:19.380704 22856 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:19.381285 22856 leader_election.cc:302] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b7e4e12855c4a018c3a32d3e284b9cb; no voters:
I20260807 18:09:19.382043 22856 leader_election.cc:288] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:09:19.382193 22862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:19.383045 22862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 LEADER]: Becoming Leader. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Running, Role: LEADER
I20260807 18:09:19.383690 22862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:19.384492 22856 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:19.386826 22863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:19.387334 22863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:19.390837 22867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:19.390681 22864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b7e4e12855c4a018c3a32d3e284b9cb. Latest consensus state: current_term: 1 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:19.391284 22864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:19.397181 22867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:19.407760 22867 catalog_manager.cc:1347] Generated new cluster ID: 567ce33278c64485afd6caf944c566bc
I20260807 18:09:19.407907 22867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:19.421870 22867 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:09:19.423336 22867 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:19.432951 22867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Generated new TSK 0
I20260807 18:09:19.433974 22867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:09:19.505878 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:19.870363 22883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:19.870748 22883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:19.870922 22883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:19.878934 22883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:19.879115 22883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:19.879292 22883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:19.890200 22883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:19.891872 22883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:19.893491 22883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:19.909267 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:19.909315 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:19.909212 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:19.910465 22883 server_base.cc:1034] running on GCE node
I20260807 18:09:19.911190 22883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:19.912266 22883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:19.913542 22883 hybrid_clock.cc:648] HybridClock initialized: now 1786126159913465 us; error 49 us; skew 500 ppm
I20260807 18:09:19.914002 22883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:19 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126159, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:19.918095 22883 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:19.919956 22883 webserver.cc:460] Webserver started at http://127.8.62.129:39943/ using document root <none> and password file <none>
I20260807 18:09:19.920902 22883 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:19.921072 22883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:19.921451 22883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:19.925015 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "83050f7723e90e60d271b8d40b81a0a6"
server_key_iv: "8ad0b85a145dbd8de483812bae0c036c"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.926002 22883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "83050f7723e90e60d271b8d40b81a0a6"
server_key_iv: "8ad0b85a145dbd8de483812bae0c036c"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.932771 22883 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 18:09:19.937000 22898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:19.938589 22883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:09:19.938812 22883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "83050f7723e90e60d271b8d40b81a0a6"
server_key_iv: "8ad0b85a145dbd8de483812bae0c036c"
server_key_version: "encryptionkey@0"
I20260807 18:09:19.939111 22883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:19.965343 22883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:19.970692 22883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:19.971065 22883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:19.972278 22883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:19.973940 22883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:19.974030 22883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:19.974202 22883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:19.974282 22883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:20.014025 22883 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:32987
I20260807 18:09:20.014128 23018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:32987 every 8 connection(s)
I20260807 18:09:20.015513 22883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:20.016124 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 22883
I20260807 18:09:20.016381 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:09:20.017369 8442 external_mini_cluster.cc:1428] Setting key a92f255d09c3244af85b92fe21ab8a8c
Aug 07 18:09:20 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126159, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:20.040649 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:20.018479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:51789 (local address 127.8.62.190:46687)
0807 18:09:20.019020 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:20.019024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:20.020324 (+ 1300us) server_negotiation.cc:407] Connection header received
0807 18:09:20.022417 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:20.022422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:20.022539 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:20.022669 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:20.024407 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.025237 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.026955 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.027439 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.027618 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:20.031992 (+ 4374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:20.032003 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:20.032011 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:20.032070 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:20.036202 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:20.037360 (+ 1158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:20.037370 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:20.037376 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:20.037562 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:20.038143 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:20.038160 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:20.038166 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:20.038596 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:20.038776 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:20.039794 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0807 18:09:20.040105 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":316,"threads_started":1}
I20260807 18:09:20.041492 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:20.019220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:51789)
0807 18:09:20.020159 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:20.020200 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:20.022100 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:20.022954 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:20.022963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:20.023550 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:20.024183 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:20.024204 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.025437 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.025441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:20.026771 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:20.026785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.027702 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.027707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:20.027950 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:20.028870 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:20.028897 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:20.031652 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:20.036438 (+ 4786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:20.036448 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:20.036465 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:20.037194 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:20.037735 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:20.037739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:20.037741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:20.037971 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:20.038789 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:20.038801 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:20.039283 (+ 482us) client_negotiation.cc:769] Sending connection context
0807 18:09:20.039619 (+ 336us) client_negotiation.cc:240] Negotiation successful
0807 18:09:20.040180 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":427,"threads_started":1}
I20260807 18:09:20.043469 23019 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:20.043859 23019 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:09:20.044823 23019 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
I20260807 18:09:20.047293 22820 ts_manager.cc:194] Registered new tserver with Master: 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987)
I20260807 18:09:20.049000 22820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:51789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:09:20.080862 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:20.439829 23025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:20.440187 23025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:20.440352 23025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:20.448362 23025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:20.448585 23025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:20.448774 23025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:20.458846 23025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:20.460354 23025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:20.461823 23025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:20.476465 23030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:20.476570 23033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:20.476497 23031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:20.482724 23025 server_base.cc:1034] running on GCE node
I20260807 18:09:20.483475 23025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:20.484697 23025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:20.485944 23025 hybrid_clock.cc:648] HybridClock initialized: now 1786126160485851 us; error 86 us; skew 500 ppm
I20260807 18:09:20.486312 23025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:20 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126160, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:20.489849 23025 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:20.491489 23025 webserver.cc:460] Webserver started at http://127.8.62.130:42911/ using document root <none> and password file <none>
I20260807 18:09:20.492399 23025 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:20.492596 23025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:20.492921 23025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:20.496064 23025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "0b0d55dba8b2447e990c5438c4299112"
format_stamp: "Formatted at 2026-08-07 18:09:20 on dist-test-slave-rnrw"
server_key: "6487df446f390e68873101430f929f6a"
server_key_iv: "b626f99def3b97eb0424075868e38696"
server_key_version: "encryptionkey@0"
I20260807 18:09:20.496943 23025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "0b0d55dba8b2447e990c5438c4299112"
format_stamp: "Formatted at 2026-08-07 18:09:20 on dist-test-slave-rnrw"
server_key: "6487df446f390e68873101430f929f6a"
server_key_iv: "b626f99def3b97eb0424075868e38696"
server_key_version: "encryptionkey@0"
I20260807 18:09:20.502439 23025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260807 18:09:20.506486 23041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:20.508045 23025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:09:20.508235 23025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "0b0d55dba8b2447e990c5438c4299112"
format_stamp: "Formatted at 2026-08-07 18:09:20 on dist-test-slave-rnrw"
server_key: "6487df446f390e68873101430f929f6a"
server_key_iv: "b626f99def3b97eb0424075868e38696"
server_key_version: "encryptionkey@0"
I20260807 18:09:20.508474 23025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:20.531462 23025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:20.536461 23025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:20.536827 23025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:20.538200 23025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:20.539692 23025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:20.539803 23025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:20.539937 23025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:20.540009 23025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:20.575184 23025 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:40531
I20260807 18:09:20.575232 23158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:40531 every 8 connection(s)
I20260807 18:09:20.576664 23025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:20.578699 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23025
I20260807 18:09:20.578898 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:09:20.579443 8442 external_mini_cluster.cc:1428] Setting key 4eadf56e45132442ad1b2b6925b8b540
Aug 07 18:09:20 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126160, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:20.601171 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:20.580382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:48861 (local address 127.8.62.190:46687)
0807 18:09:20.580907 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:20.580913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:20.581893 (+ 980us) server_negotiation.cc:407] Connection header received
0807 18:09:20.583902 (+ 2009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:20.583907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:20.584008 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:20.584131 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:20.585854 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.586630 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.588344 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.588856 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.588997 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:20.594230 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:20.594235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:20.594239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:20.594278 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:20.597168 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:20.598399 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:20.598404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:20.598407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:20.598558 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:20.599133 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:20.599136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:20.599138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:20.599419 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:20.599539 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:20.600688 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0807 18:09:20.600896 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":292,"threads_started":1}
I20260807 18:09:20.602945 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:20.580811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:48861)
0807 18:09:20.581742 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:20.581781 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:20.583605 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:20.584383 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:20.584392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:20.585033 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:20.585661 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:20.585691 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.586815 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.586820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:20.588169 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:20.588184 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:20.589125 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:20.589133 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:20.589380 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:20.590138 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:20.590164 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:20.593911 (+ 3747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:20.597382 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:20.597396 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:20.597418 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:20.598222 (+ 804us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:20.598702 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:20.598710 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:20.598716 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:20.599005 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:20.599581 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:20.599593 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:20.600128 (+ 535us) client_negotiation.cc:769] Sending connection context
0807 18:09:20.600476 (+ 348us) client_negotiation.cc:240] Negotiation successful
0807 18:09:20.601210 (+ 734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":454,"threads_started":1}
I20260807 18:09:20.604735 23159 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:20.605141 23159 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:20.606169 23159 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
I20260807 18:09:20.608289 22820 ts_manager.cc:194] Registered new tserver with Master: 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531)
I20260807 18:09:20.609305 22820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:48861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:09:20.645280 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:21.006111 23165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:21.006441 23165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:21.006636 23165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:21.015455 23165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:21.015609 23165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:21.015805 23165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:21.027797 23165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:21.029224 23165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:21.030566 23165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:21.045246 23170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:21.045284 23171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:21.045207 23173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:21.046762 23165 server_base.cc:1034] running on GCE node
I20260807 18:09:21.047513 23165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:21.048682 23165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:21.049944 23165 hybrid_clock.cc:648] HybridClock initialized: now 1786126161049886 us; error 52 us; skew 500 ppm
I20260807 18:09:21.050319 23165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:09:21.053339 23019 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
Aug 07 18:09:21 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126161, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:21.054388 23165 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:21.056356 23165 webserver.cc:460] Webserver started at http://127.8.62.131:42003/ using document root <none> and password file <none>
I20260807 18:09:21.057163 23165 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:21.057281 23165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:21.057546 23165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:21.060691 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
format_stamp: "Formatted at 2026-08-07 18:09:21 on dist-test-slave-rnrw"
server_key: "1d121f3e61e2f09f0bdafc9ae985b0c7"
server_key_iv: "1d6f39163906be929afb910d6a526a57"
server_key_version: "encryptionkey@0"
I20260807 18:09:21.061587 23165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
format_stamp: "Formatted at 2026-08-07 18:09:21 on dist-test-slave-rnrw"
server_key: "1d121f3e61e2f09f0bdafc9ae985b0c7"
server_key_iv: "1d6f39163906be929afb910d6a526a57"
server_key_version: "encryptionkey@0"
I20260807 18:09:21.067510 23165 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:09:21.071630 23180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:21.073289 23165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:09:21.073477 23165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
format_stamp: "Formatted at 2026-08-07 18:09:21 on dist-test-slave-rnrw"
server_key: "1d121f3e61e2f09f0bdafc9ae985b0c7"
server_key_iv: "1d6f39163906be929afb910d6a526a57"
server_key_version: "encryptionkey@0"
I20260807 18:09:21.073706 23165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:21.093869 23165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:21.098531 23165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:21.098860 23165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:21.100003 23165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:21.101603 23165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:21.101691 23165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:21.101837 23165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:21.101890 23165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:21.138687 23165 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:37355
I20260807 18:09:21.138734 23298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:37355 every 8 connection(s)
I20260807 18:09:21.140141 23165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:21.145663 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23165
I20260807 18:09:21.145857 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:09:21.146353 8442 external_mini_cluster.cc:1428] Setting key 373835144bc8dab521f0d6b0c3af9aed
Aug 07 18:09:21 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126161, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:21.163556 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.143068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:48949 (local address 127.8.62.190:46687)
0807 18:09:21.143715 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.143740 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.144811 (+ 1071us) server_negotiation.cc:407] Connection header received
0807 18:09:21.146929 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.146935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.147058 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.147214 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:21.149098 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.150121 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.151859 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.152287 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.152436 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.156521 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:21.156527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:21.156530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:21.156624 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:21.159608 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.160674 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.160680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.160683 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.160867 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.161544 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.161548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.161550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.161854 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:21.162012 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.163025 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.163241 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":316,"threads_started":1}
I20260807 18:09:21.164670 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.143750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:48949)
0807 18:09:21.144581 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.144631 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.146599 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.147521 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.147531 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.148124 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:21.148886 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.148909 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.150322 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.150326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.151679 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.151698 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.152609 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.152617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.152872 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.153675 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:21.153708 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:21.156233 (+ 2525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:21.159825 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:21.159836 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:21.159863 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:21.160456 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:21.161053 (+ 597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:21.161058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:21.161061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:21.161390 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:21.162041 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:21.162052 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:21.162545 (+ 493us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.162923 (+ 378us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.163396 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":283,"threads_started":1}
I20260807 18:09:21.166412 23299 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:21.166842 23299 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:21.167764 23299 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
I20260807 18:09:21.169622 22820 ts_manager.cc:194] Registered new tserver with Master: 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355)
I20260807 18:09:21.170527 22820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:48949
I20260807 18:09:21.176035 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:09:21.196825 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.181601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.8.62.190:46687)
0807 18:09:21.181863 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.181871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.182312 (+ 441us) server_negotiation.cc:407] Connection header received
0807 18:09:21.182621 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.182626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.182700 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.182811 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:21.184306 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.185208 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.186620 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.187143 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.187289 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.190214 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:21.190220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:21.190223 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:21.190268 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:21.193673 (+ 3405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.194535 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.194541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.194547 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.194712 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.195157 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.195165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.195168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.195555 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:21.195833 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.196181 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.196447 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:09:21.201720 22820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:09:21.208860 22820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:09:21.242647 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.223862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.8.62.131:37355)
0807 18:09:21.224952 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.224966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.233404 (+ 8438us) server_negotiation.cc:407] Connection header received
0807 18:09:21.233628 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.233641 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.234052 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.234309 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:21.235190 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.236678 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.239274 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.241211 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.241341 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.242034 (+ 693us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.242117 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.242282 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":538,"threads_started":1}
I20260807 18:09:21.242817 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.222941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55454)
0807 18:09:21.228775 (+ 5834us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.228848 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.229085 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.235015 (+ 5930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.235020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.235060 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:21.235587 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.235604 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.237881 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.237890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.239851 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.239865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.241886 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.241892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.242068 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.242106 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.242180 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.242333 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5416,"mutex_wait_us":5,"thread_start_us":10764,"threads_started":1}
I20260807 18:09:21.243158 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.223134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:37355 (local address 127.0.0.1:40290)
0807 18:09:21.233208 (+ 10074us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.233234 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.233412 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.234476 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.234485 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.234525 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:21.234930 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.234944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.236817 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.236823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.239052 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.239077 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.241549 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.241558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.241817 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.241889 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.242025 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.242256 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9893,"mutex_wait_us":30,"thread_start_us":9687,"threads_started":1}
I20260807 18:09:21.243361 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.228271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42310 (local address 127.8.62.130:40531)
0807 18:09:21.231671 (+ 3400us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.231681 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.234267 (+ 2586us) server_negotiation.cc:407] Connection header received
0807 18:09:21.234388 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.234396 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.234651 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.234883 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:21.235926 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.237562 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.240231 (+ 2669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.241705 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.241851 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.242060 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.242819 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.243023 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3206,"thread_start_us":3087,"threads_started":1}
I20260807 18:09:21.243873 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.227768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40531 (local address 127.0.0.1:42310)
0807 18:09:21.233967 (+ 6199us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.233986 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.234118 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.235180 (+ 1062us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.235186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.235213 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:21.235741 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.235753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.237728 (+ 1975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.237734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.240055 (+ 2321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.240071 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.242213 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.242219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.242381 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.242413 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.242508 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.242650 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6069,"thread_start_us":755,"threads_started":1}
I20260807 18:09:21.244272 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.223147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.8.62.129:32987)
0807 18:09:21.234468 (+ 11321us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.234477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.234501 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:09:21.234582 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.234588 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.234791 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.235851 (+ 1060us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:21.236165 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.237741 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.240029 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.241611 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.243415 (+ 1804us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.243658 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.243727 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.243894 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11147,"thread_start_us":10989,"threads_started":1}
I20260807 18:09:21.247385 23230 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 (DEFAULT_TABLE table=test-table [id=f7c17233223c4d1a957e97d4c70aebd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:21.247757 23093 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 (DEFAULT_TABLE table=test-table [id=f7c17233223c4d1a957e97d4c70aebd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:21.247772 22949 tablet_service.cc:1467] Processing CreateTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 (DEFAULT_TABLE table=test-table [id=f7c17233223c4d1a957e97d4c70aebd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:21.248978 23230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:21.249203 23093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:21.249202 22949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:21.261189 23320 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Bootstrap starting.
I20260807 18:09:21.265365 23321 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Bootstrap starting.
I20260807 18:09:21.265767 23320 tablet_bootstrap.cc:654] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:21.267212 23320 log.cc:824] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:21.268239 23322 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Bootstrap starting.
I20260807 18:09:21.269811 23321 tablet_bootstrap.cc:654] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:21.271401 23321 log.cc:824] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:21.272324 23322 tablet_bootstrap.cc:654] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:21.273545 23322 log.cc:824] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:21.273707 23320 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: No bootstrap required, opened a new log
I20260807 18:09:21.274084 23320 ts_tablet_manager.cc:1397] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260807 18:09:21.275806 23321 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: No bootstrap required, opened a new log
I20260807 18:09:21.276109 23321 ts_tablet_manager.cc:1397] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 18:09:21.279980 23322 tablet_bootstrap.cc:492] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: No bootstrap required, opened a new log
I20260807 18:09:21.280310 23322 ts_tablet_manager.cc:1397] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260807 18:09:21.285960 23320 raft_consensus.cc:348] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.286525 23320 raft_consensus.cc:374] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:21.286696 23320 raft_consensus.cc:729] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02954b36666e4e8dab6cdc5dc97a6b4a, State: Initialized, Role: FOLLOWER
I20260807 18:09:21.287468 23320 consensus_queue.cc:260] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.287505 23321 raft_consensus.cc:348] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.288041 23321 raft_consensus.cc:374] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:21.288849 23322 raft_consensus.cc:348] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.289479 23321 raft_consensus.cc:729] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d55dba8b2447e990c5438c4299112, State: Initialized, Role: FOLLOWER
I20260807 18:09:21.289722 23322 raft_consensus.cc:374] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:21.289867 23322 raft_consensus.cc:729] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad6aed6744d4ec7a2fb8e41fadb9b29, State: Initialized, Role: FOLLOWER
I20260807 18:09:21.290560 23322 consensus_queue.cc:260] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.290653 23321 consensus_queue.cc:260] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.292783 23322 ts_tablet_manager.cc:1428] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Time spent starting tablet: real 0.012s user 0.010s sys 0.004s
I20260807 18:09:21.295367 23159 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
I20260807 18:09:21.296437 23321 ts_tablet_manager.cc:1428] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
I20260807 18:09:21.297621 23299 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
I20260807 18:09:21.298462 23320 ts_tablet_manager.cc:1428] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Time spent starting tablet: real 0.024s user 0.009s sys 0.013s
W20260807 18:09:21.328961 23160 tablet.cc:2369] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:21.371747 23326 raft_consensus.cc:482] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:09:21.372138 23326 raft_consensus.cc:504] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.374590 23326 leader_election.cc:288] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987), 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531)
I20260807 18:09:21.382217 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.375049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.8.62.131:41053)
0807 18:09:21.375497 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.375520 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.375752 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.376133 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.376138 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.376165 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:21.376360 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.376369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.377692 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.377700 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.379718 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.379752 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.381319 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.381331 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.381559 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.381601 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.381717 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.381868 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":3}
I20260807 18:09:21.382287 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.375201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:41053 (local address 127.8.62.129:32987)
0807 18:09:21.375440 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.375450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.375633 (+ 183us) server_negotiation.cc:407] Connection header received
0807 18:09:21.375919 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.375929 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.375999 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.376108 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:21.376514 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.377562 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.379930 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.381153 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.381281 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.381343 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.381793 (+ 450us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.381998 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:09:21.382990 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.375144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:38455 (local address 127.8.62.130:40531)
0807 18:09:21.375491 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.375498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.376243 (+ 745us) server_negotiation.cc:407] Connection header received
0807 18:09:21.376466 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.376470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.376590 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.376705 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:21.377300 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.378262 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.380717 (+ 2455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.381809 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.381960 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.382043 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.382589 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.382774 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260807 18:09:21.383190 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.375220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40531 (local address 127.8.62.131:38455)
0807 18:09:21.376119 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.376134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:21.376311 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:21.376719 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:21.376724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:21.376749 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:21.377142 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.377155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.378422 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.378431 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.380526 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:21.380568 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.382066 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.382074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:21.382304 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.382350 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:09:21.382534 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 18:09:21.382696 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":7,"thread_start_us":510,"threads_started":1}
I20260807 18:09:21.383689 22969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3bfdc5fae74c3fa0aac19c0385cf34" candidate_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" is_pre_election: true
I20260807 18:09:21.384245 23113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3bfdc5fae74c3fa0aac19c0385cf34" candidate_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0d55dba8b2447e990c5438c4299112" is_pre_election: true
I20260807 18:09:21.384348 22969 raft_consensus.cc:2454] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02954b36666e4e8dab6cdc5dc97a6b4a in term 0.
I20260807 18:09:21.384815 23113 raft_consensus.cc:2454] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02954b36666e4e8dab6cdc5dc97a6b4a in term 0.
I20260807 18:09:21.385295 23181 leader_election.cc:302] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02954b36666e4e8dab6cdc5dc97a6b4a, 5ad6aed6744d4ec7a2fb8e41fadb9b29; no voters:
I20260807 18:09:21.385797 23326 raft_consensus.cc:2781] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:09:21.386011 23326 raft_consensus.cc:482] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:09:21.386133 23326 raft_consensus.cc:3037] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:21.388743 23326 raft_consensus.cc:504] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.389539 23326 leader_election.cc:288] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [CANDIDATE]: Term 1 election: Requested vote from peers 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987), 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531)
I20260807 18:09:21.390295 22969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3bfdc5fae74c3fa0aac19c0385cf34" candidate_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
I20260807 18:09:21.390290 23113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a3bfdc5fae74c3fa0aac19c0385cf34" candidate_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b0d55dba8b2447e990c5438c4299112"
I20260807 18:09:21.390578 22969 raft_consensus.cc:3037] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:21.390632 23113 raft_consensus.cc:3037] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Advancing to term 1
W20260807 18:09:21.391912 23300 tablet.cc:2369] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:21.394471 22969 raft_consensus.cc:2454] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02954b36666e4e8dab6cdc5dc97a6b4a in term 1.
I20260807 18:09:21.394493 23113 raft_consensus.cc:2454] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02954b36666e4e8dab6cdc5dc97a6b4a in term 1.
I20260807 18:09:21.395227 23182 leader_election.cc:302] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02954b36666e4e8dab6cdc5dc97a6b4a, 0b0d55dba8b2447e990c5438c4299112; no voters:
I20260807 18:09:21.395777 23326 raft_consensus.cc:2781] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:21.396804 23326 raft_consensus.cc:686] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 LEADER]: Becoming Leader. State: Replica: 02954b36666e4e8dab6cdc5dc97a6b4a, State: Running, Role: LEADER
I20260807 18:09:21.397373 23326 consensus_queue.cc:237] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } }
I20260807 18:09:21.402850 22819 catalog_manager.cc:5495] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131). New cstate: current_term: 1 leader_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } health_report { overall_health: UNKNOWN } } }
I20260807 18:09:21.433308 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.426963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.8.62.131:37355)
0807 18:09:21.427226 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.427232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.427250 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:09:21.427343 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.427348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.427456 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.427619 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:09:21.428175 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.428968 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.430605 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.431072 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.431185 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.432047 (+ 862us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:09:21.432642 (+ 595us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:09:21.432759 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.432903 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.432982 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260807 18:09:21.442760 22969 raft_consensus.cc:1261] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Refusing update from remote peer 02954b36666e4e8dab6cdc5dc97a6b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:21.443056 23113 raft_consensus.cc:1261] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Refusing update from remote peer 02954b36666e4e8dab6cdc5dc97a6b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:21.444294 23326 consensus_queue.cc:1035] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:21.444675 23330 consensus_queue.cc:1035] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:21.476634 22820 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:21.476969 22820 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:21.481848 22820 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Sending DeleteTablet for 3 replicas of tablet 5a3bfdc5fae74c3fa0aac19c0385cf34
I20260807 18:09:21.483348 22949 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:21 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:55454
I20260807 18:09:21.483767 23093 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:21 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42310
I20260807 18:09:21.483875 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22883
I20260807 18:09:21.484253 23230 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:21 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40290
I20260807 18:09:21.497660 23344 tablet_replica.cc:330] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: stopping tablet replica
I20260807 18:09:21.498251 23344 raft_consensus.cc:2229] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:21.498857 23344 raft_consensus.cc:2258] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:21.501536 23344 ts_tablet_manager.cc:1905] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:21.509325 23344 ts_tablet_manager.cc:1918] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:21.509373 23345 tablet_replica.cc:330] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: stopping tablet replica
I20260807 18:09:21.509541 23344 log.cc:1196] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/5a3bfdc5fae74c3fa0aac19c0385cf34
I20260807 18:09:21.509971 23345 raft_consensus.cc:2229] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:09:21.510193 23344 ts_tablet_manager.cc:1939] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 0b0d55dba8b2447e990c5438c4299112: Deleting consensus metadata
I20260807 18:09:21.510807 23345 raft_consensus.cc:2258] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:21.511829 23343 tablet_replica.cc:330] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: stopping tablet replica
Aug 07 18:09:21 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:09:21.512317 22804 catalog_manager.cc:4841] TS 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531): tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 (table test-table [id=f7c17233223c4d1a957e97d4c70aebd9]) successfully deleted
I20260807 18:09:21.512336 23343 raft_consensus.cc:2229] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:21.513485 23343 raft_consensus.cc:2258] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:21.516212 23343 ts_tablet_manager.cc:1905] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:21.516218 23345 ts_tablet_manager.cc:1905] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:21.523672 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.485271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55456 (local address 127.8.62.129:32987)
0807 18:09:21.485486 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.485492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.497284 (+ 11792us) server_negotiation.cc:407] Connection header received
0807 18:09:21.497706 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.497716 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.497871 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.498024 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:21.500156 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.501816 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.503961 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.505021 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.505217 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.513887 (+ 8670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:21.513906 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:21.513925 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:21.513982 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:21.518616 (+ 4634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.519778 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.519788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.519792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.519970 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.520494 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.520500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.520503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.521048 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:21.522714 (+ 1666us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.522918 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.523218 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:09:21.524785 23013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:21.525238 23345 ts_tablet_manager.cc:1918] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:21.525609 23345 log.cc:1196] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/5a3bfdc5fae74c3fa0aac19c0385cf34
I20260807 18:09:21.526515 23345 ts_tablet_manager.cc:1939] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting consensus metadata
I20260807 18:09:21.529421 22806 catalog_manager.cc:4841] TS 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355): tablet 5a3bfdc5fae74c3fa0aac19c0385cf34 (table test-table [id=f7c17233223c4d1a957e97d4c70aebd9]) successfully deleted
I20260807 18:09:21.639288 23343 ts_tablet_manager.cc:1918] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:21.639998 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22883
W20260807 18:09:21.660984 22803 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.62.129:32987 (error 108)
I20260807 18:09:21.661317 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23025
I20260807 18:09:21.663612 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.663069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55460)
0807 18:09:21.663278 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.663416 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 18:09:21.663903 22803 catalog_manager.cc:4584] TS 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a3bfdc5fae74c3fa0aac19c0385cf34: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Aug 07 18:09:21 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:09:21.680296 23319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.662310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.8.62.130:40531)
0807 18:09:21.662522 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.662528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.662560 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:09:21.662696 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.662702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.662797 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.662924 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:21.664506 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.665795 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.667300 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.667976 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.668122 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.672895 (+ 4773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:21.672916 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:21.672937 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:21.672985 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:21.676839 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.677747 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.677756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.677758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.677959 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.678471 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.678480 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.678483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.679106 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:21.679427 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.679644 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.679870 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 18:09:21.681092 23154 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:21.717876 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.717335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55472)
0807 18:09:21.717592 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.717699 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:09:21.719062 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.718681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55478)
0807 18:09:21.718838 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.718921 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 18:09:21.779793 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.779276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55480)
0807 18:09:21.779485 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.779635 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 18:09:21.781013 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.780645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55496)
0807 18:09:21.780804 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.780890 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:09:21.832566 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23025
I20260807 18:09:21.858469 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23165
I20260807 18:09:21.861701 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.861108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55500)
0807 18:09:21.861347 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.861464 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:09:21.863040 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.862551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55516)
0807 18:09:21.862776 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:21.862875 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:32987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
Aug 07 18:09:21 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:09:21.880309 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:21.859576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.8.62.131:37355)
0807 18:09:21.859783 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:21.859790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:21.859944 (+ 154us) server_negotiation.cc:407] Connection header received
0807 18:09:21.860074 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:21.860080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:21.860177 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:21.860316 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:21.862014 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.863384 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.865525 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:21.865998 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:21.866140 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:21.871925 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:21.871941 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:21.871958 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:21.872006 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:21.876313 (+ 4307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.877394 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.877406 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.877410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.877645 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:21.878314 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:21.878321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:21.878324 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:21.878851 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:21.879263 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:21.879622 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:09:21.879894 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:09:21.881186 23294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:22.010327 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23165
I20260807 18:09:22.029865 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 22787
I20260807 18:09:22.031215 22851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:22.146816 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 22787
I20260807 18:09:22.172273 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--webserver_interface=127.8.62.190
--webserver_port=41707
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:22.557546 23352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:22.557891 23352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:22.558072 23352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:22.566039 23352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:22.566182 23352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:22.566267 23352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:22.566357 23352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:22.566449 23352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:22.577562 23352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:22.579193 23352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:22.580797 23352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:22.608954 23360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:22.608860 23361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:22.608888 23363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:22.610067 23352 server_base.cc:1034] running on GCE node
I20260807 18:09:22.611382 23352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:22.613952 23352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:22.615265 23352 hybrid_clock.cc:648] HybridClock initialized: now 1786126162615206 us; error 73 us; skew 500 ppm
I20260807 18:09:22.615768 23352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:22 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126162, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:22.626982 23352 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:22.629501 23352 webserver.cc:460] Webserver started at http://127.8.62.190:41707/ using document root <none> and password file <none>
I20260807 18:09:22.630574 23352 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:22.630765 23352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:22.637962 23352 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:09:22.641752 23370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:22.643726 23352 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:09:22.643975 23352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "9b7e4e12855c4a018c3a32d3e284b9cb"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "a5017e4e83223f60850c6f51b522980e"
server_key_iv: "298618ffb9db815e15b6c7074ffb17b3"
server_key_version: "encryptionkey@0"
I20260807 18:09:22.645290 23352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:22.673070 23352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:22.679100 23352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:22.679522 23352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:22.702761 23352 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46687
I20260807 18:09:22.702811 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46687 every 8 connection(s)
I20260807 18:09:22.704631 23352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:22.712805 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23352
I20260807 18:09:22.712867 23426 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:09:22.713708 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:32987
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=39943
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:22.716964 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap starting.
I20260807 18:09:22.739354 23426 log.cc:824] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:22.753417 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:09:22.754140 23426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap complete.
I20260807 18:09:22.764250 23426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:22.765534 23426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Initialized, Role: FOLLOWER
I20260807 18:09:22.766258 23426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:22.766564 23426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:22.766721 23426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:22.766929 23426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:09:22.771409 23426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:22.772081 23426 leader_election.cc:302] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b7e4e12855c4a018c3a32d3e284b9cb; no voters:
I20260807 18:09:22.772843 23426 leader_election.cc:288] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:09:22.772994 23430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:09:22.774384 23430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 LEADER]: Becoming Leader. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Running, Role: LEADER
I20260807 18:09:22.774986 23430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:22.775642 23426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:22.784776 23432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b7e4e12855c4a018c3a32d3e284b9cb. Latest consensus state: current_term: 2 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:22.785336 23432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:22.784777 23431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:22.785858 23431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:22.793742 23437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:22.805505 23437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:22.807606 23437 catalog_manager.cc:1259] Loaded cluster ID: 567ce33278c64485afd6caf944c566bc
I20260807 18:09:22.807754 23437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:22.812340 23437 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 18:09:22.835791 23437 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 18:09:22.854130 23437 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:22.856536 23437 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Loaded TSK: 0
I20260807 18:09:22.856491 23449 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:09:22.858201 23437 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:09:23.133469 23428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:23.133812 23428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:23.133966 23428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:23.142123 23428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:23.142247 23428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:23.142437 23428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:23.153609 23428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:32987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=39943
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:23.155053 23428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:23.156494 23428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:23.175251 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:23.175325 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:23.175287 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:23.175853 23428 server_base.cc:1034] running on GCE node
I20260807 18:09:23.177136 23428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:23.178748 23428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:23.180044 23428 hybrid_clock.cc:648] HybridClock initialized: now 1786126163179964 us; error 74 us; skew 500 ppm
I20260807 18:09:23.180497 23428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:23 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126163, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:23.185261 23428 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:23.187390 23428 webserver.cc:460] Webserver started at http://127.8.62.129:39943/ using document root <none> and password file <none>
I20260807 18:09:23.188393 23428 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:23.188581 23428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:23.196784 23428 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 18:09:23.201229 23466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:23.203382 23428 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:09:23.203612 23428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "83050f7723e90e60d271b8d40b81a0a6"
server_key_iv: "8ad0b85a145dbd8de483812bae0c036c"
server_key_version: "encryptionkey@0"
I20260807 18:09:23.205084 23428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:23.233711 23428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:23.239552 23428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:23.239892 23428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:23.241413 23428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:23.244131 23474 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 18:09:23.252278 23474 ts_tablet_manager.cc:1852] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 18:09:23.253448 23474 ts_tablet_manager.cc:1905] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:23.261729 23474 ts_tablet_manager.cc:1918] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:23.262006 23474 log.cc:1196] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/5a3bfdc5fae74c3fa0aac19c0385cf34
I20260807 18:09:23.262537 23474 ts_tablet_manager.cc:1939] T 5a3bfdc5fae74c3fa0aac19c0385cf34 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting consensus metadata
I20260807 18:09:23.263054 23428 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 18:09:23.263183 23428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.020s user 0.001s sys 0.000s
I20260807 18:09:23.263290 23428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:23.263355 23428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:23.304225 23428 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:32987
I20260807 18:09:23.304279 23588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:32987 every 8 connection(s)
I20260807 18:09:23.305719 23428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:23.315697 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23428
I20260807 18:09:23.316412 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:40531
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=42911
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:09:23 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126163, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:23.334342 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:23.309151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:58105 (local address 127.8.62.190:46687)
0807 18:09:23.309732 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:23.309745 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:23.310542 (+ 797us) server_negotiation.cc:407] Connection header received
0807 18:09:23.313227 (+ 2685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:23.313241 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:23.313622 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:23.314034 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:23.315726 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.317225 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.318938 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.319657 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.319887 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:23.324172 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:23.324181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:23.324199 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:23.324254 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:23.328715 (+ 4461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:23.329755 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:23.329773 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:23.329780 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:23.329980 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:23.330520 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:23.330528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:23.330532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:23.331115 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:23.331354 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:23.332311 (+ 957us) server_negotiation.cc:299] Negotiation successful
0807 18:09:23.332830 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":255,"threads_started":1}
I20260807 18:09:23.334326 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:23.309485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:58105)
0807 18:09:23.310354 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:23.310396 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:23.312622 (+ 2226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:23.314170 (+ 1548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:23.314183 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:23.314872 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:23.315486 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:23.315507 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.317426 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.317430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:23.318760 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:23.318777 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.319932 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.319940 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:23.320226 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:23.321097 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:23.321124 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:23.323816 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:23.328926 (+ 5110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:23.328938 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:23.328960 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:23.329571 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:23.330139 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:23.330144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:23.330149 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:23.330380 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:23.331288 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:23.331297 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:23.331722 (+ 425us) client_negotiation.cc:769] Sending connection context
0807 18:09:23.332041 (+ 319us) client_negotiation.cc:240] Negotiation successful
0807 18:09:23.332754 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":287,"threads_started":1}
I20260807 18:09:23.338358 23589 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:23.338776 23589 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:23.339598 23589 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
I20260807 18:09:23.342432 23388 ts_manager.cc:194] Registered new tserver with Master: 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987)
I20260807 18:09:23.344113 23388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:58105
W20260807 18:09:23.678251 23593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:23.678608 23593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:23.678778 23593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:23.686401 23593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:23.686589 23593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:23.686779 23593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:23.697842 23593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:40531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=42911
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:23.699245 23593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:23.700773 23593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:23.715615 23601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:23.716281 23603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:23.716732 23593 server_base.cc:1034] running on GCE node
W20260807 18:09:23.716833 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:23.717628 23593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:23.718788 23593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:23.720058 23593 hybrid_clock.cc:648] HybridClock initialized: now 1786126163720001 us; error 45 us; skew 500 ppm
I20260807 18:09:23.720453 23593 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:23 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126163, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:23.724200 23593 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:23.726052 23593 webserver.cc:460] Webserver started at http://127.8.62.130:42911/ using document root <none> and password file <none>
I20260807 18:09:23.726855 23593 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:23.726982 23593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:23.732890 23593 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:23.736128 23610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:23.738016 23593 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:23.738263 23593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "0b0d55dba8b2447e990c5438c4299112"
format_stamp: "Formatted at 2026-08-07 18:09:20 on dist-test-slave-rnrw"
server_key: "6487df446f390e68873101430f929f6a"
server_key_iv: "b626f99def3b97eb0424075868e38696"
server_key_version: "encryptionkey@0"
I20260807 18:09:23.739506 23593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:23.766957 23593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:23.771678 23593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:23.772092 23593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:23.773276 23593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:23.774816 23593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:23.774936 23593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:23.775077 23593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:23.775172 23593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:23.811210 23593 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:40531
I20260807 18:09:23.811264 23730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:40531 every 8 connection(s)
I20260807 18:09:23.812873 23593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:23.815044 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23593
I20260807 18:09:23.815757 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:37355
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=42003
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:09:23 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126163, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:23.839413 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:23.815812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:48261 (local address 127.8.62.190:46687)
0807 18:09:23.816261 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:23.816269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:23.817437 (+ 1168us) server_negotiation.cc:407] Connection header received
0807 18:09:23.820465 (+ 3028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:23.820522 (+ 57us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:23.820708 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:23.820807 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:23.823221 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.824026 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.826176 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.827032 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.827206 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:23.831448 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:23.831458 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:23.831467 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:23.831535 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:23.835462 (+ 3927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:23.836359 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:23.836368 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:23.836374 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:23.836585 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:23.837095 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:23.837104 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:23.837109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:23.837558 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:23.837769 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:23.838580 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 18:09:23.838895 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260807 18:09:23.839998 23733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:23.816270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:48261)
0807 18:09:23.817221 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:23.817288 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:23.820082 (+ 2794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:23.821132 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:23.821147 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:23.822063 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:23.822969 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:23.823000 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.824228 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.824236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:23.825961 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:23.825976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:23.827286 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:23.827292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:23.827528 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:23.828453 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:23.828479 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:23.831173 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:23.835651 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:23.835660 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:23.835676 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:23.836197 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:23.836716 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:23.836719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:23.836721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:23.836941 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:23.837730 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:23.837737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:23.838106 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 18:09:23.838377 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 18:09:23.838813 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":373,"threads_started":1}
I20260807 18:09:23.841773 23731 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:23.842190 23731 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:23.843228 23731 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
I20260807 18:09:23.845290 23388 ts_manager.cc:194] Registered new tserver with Master: 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531)
I20260807 18:09:23.846244 23388 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:48261
W20260807 18:09:24.187497 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:24.187875 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:24.188060 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:24.195876 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:24.196055 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:24.196254 23734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:24.206882 23734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:37355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=42003
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:24.208310 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:24.209738 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:24.223795 23740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:24.223903 23742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:24.223906 23739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:24.225414 23734 server_base.cc:1034] running on GCE node
I20260807 18:09:24.226224 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:24.227360 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:24.228732 23734 hybrid_clock.cc:648] HybridClock initialized: now 1786126164228677 us; error 50 us; skew 500 ppm
I20260807 18:09:24.229089 23734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:24 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126164, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:24.232424 23734 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:24.234210 23734 webserver.cc:460] Webserver started at http://127.8.62.131:42003/ using document root <none> and password file <none>
I20260807 18:09:24.234912 23734 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:24.235029 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:24.241400 23734 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:24.244711 23749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:24.246389 23734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 18:09:24.246618 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
format_stamp: "Formatted at 2026-08-07 18:09:21 on dist-test-slave-rnrw"
server_key: "1d121f3e61e2f09f0bdafc9ae985b0c7"
server_key_iv: "1d6f39163906be929afb910d6a526a57"
server_key_version: "encryptionkey@0"
I20260807 18:09:24.247709 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:24.270123 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:24.275007 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:24.275311 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:24.276468 23734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:24.278039 23734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:24.278134 23734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:24.278216 23734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:24.278264 23734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:24.315037 23734 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:37355
I20260807 18:09:24.315142 23865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:37355 every 8 connection(s)
I20260807 18:09:24.316706 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:24.317695 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23734
Aug 07 18:09:24 dist-test-slave-rnrw krb5kdc[22773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126164, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:24.337729 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.318900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.8.62.190:46687)
0807 18:09:24.319161 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.319170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.319616 (+ 446us) server_negotiation.cc:407] Connection header received
0807 18:09:24.319857 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.319869 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.319966 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.320091 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:24.323333 (+ 3242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.324353 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.326642 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.327317 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.327450 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.331284 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:24.331292 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:24.331299 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:24.331358 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:24.334596 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.335470 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.335475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.335477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.335616 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.336065 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.336068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.336070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.336430 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:24.336807 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.337050 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.337269 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:09:24.343454 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.320002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:37849 (local address 127.8.62.190:46687)
0807 18:09:24.320966 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.320974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.323060 (+ 2086us) server_negotiation.cc:407] Connection header received
0807 18:09:24.325154 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.325159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.325242 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.325328 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:24.327612 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.328440 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.331228 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.332061 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.332230 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.336386 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:24.336393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:24.336398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:24.336430 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:24.339993 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.340946 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.340950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.340952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.341067 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.341617 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.341621 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.341623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.341934 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:24.342066 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.342902 (+ 836us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.343091 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":828,"thread_start_us":704,"threads_started":1}
I20260807 18:09:24.344712 23870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.320589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:37849)
0807 18:09:24.321980 (+ 1391us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.322058 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.324691 (+ 2633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.325673 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.325688 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.326607 (+ 919us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:24.327339 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.327360 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.328654 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.328660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.331020 (+ 2360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.331038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.332406 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.332416 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.332781 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.333477 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:24.333523 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:24.336146 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:24.340150 (+ 4004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:24.340160 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:24.340185 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:24.340794 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:24.341226 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:24.341230 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:24.341233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:24.341460 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:24.342093 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:24.342100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:24.342464 (+ 364us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.342773 (+ 309us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.343202 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":378,"threads_started":1}
I20260807 18:09:24.346611 23866 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46687
I20260807 18:09:24.346947 23866 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:24.347759 23866 heartbeater.cc:511] Master 127.8.62.190:46687 requested a full tablet report, sending...
I20260807 18:09:24.348297 23589 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
I20260807 18:09:24.349776 23387 ts_manager.cc:194] Registered new tserver with Master: 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355)
I20260807 18:09:24.350839 23387 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:37849
I20260807 18:09:24.353645 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:09:24.361150 23869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.354618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46194 (local address 127.8.62.190:46687)
0807 18:09:24.354801 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.354805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.355227 (+ 422us) server_negotiation.cc:407] Connection header received
0807 18:09:24.355431 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.355436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.355577 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.355717 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:09:24.356211 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.357127 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.358699 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.359143 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.359258 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.359823 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:09:24.360401 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:09:24.360531 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.360855 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.360945 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:09:24.362264 23387 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:46194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:09:24.369938 23387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:09:24.412740 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.8.62.130:40531)
0807 18:09:24.402459 (+ 19044us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.402470 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.402503 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:09:24.403471 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.403479 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.403727 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.403970 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:24.404675 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.406132 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.409169 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.410807 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.410942 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.411168 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.412218 (+ 1050us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.412403 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18871,"thread_start_us":19024,"threads_started":1}
I20260807 18:09:24.412740 23881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:37355 (local address 127.0.0.1:40332)
0807 18:09:24.401226 (+ 17909us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.401251 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.401404 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.403452 (+ 2048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.403460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.403490 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:24.403969 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.403984 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.406468 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.406474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.409119 (+ 2645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.409140 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.411498 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.411504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.411755 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.411914 (+ 159us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.412052 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.412276 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17746,"mutex_wait_us":114,"thread_start_us":19541,"threads_started":1}
I20260807 18:09:24.413225 23877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.0.0.1:55526)
0807 18:09:24.403140 (+ 20101us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.403160 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.403324 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.404204 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.404213 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.404237 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:24.404737 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.404754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.408036 (+ 3282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.408042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.409871 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.409883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.412282 (+ 2399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.412288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.412450 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.412474 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.412632 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.412781 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19955,"mutex_wait_us":1045,"thread_start_us":16809,"threads_started":1}
I20260807 18:09:24.413262 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55526 (local address 127.8.62.129:32987)
0807 18:09:24.402011 (+ 18190us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.402023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.403290 (+ 1267us) server_negotiation.cc:407] Connection header received
0807 18:09:24.403467 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.403475 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.403727 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.403928 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:24.404968 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.407855 (+ 2887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.410096 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.411733 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.411862 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.412118 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.412690 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.412850 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17987,"thread_start_us":17858,"threads_started":1}
I20260807 18:09:24.413673 23876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40531 (local address 127.0.0.1:42318)
0807 18:09:24.398064 (+ 15025us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.398141 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.398414 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.403997 (+ 5583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.404004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.404038 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:24.404467 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.404480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.406302 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.406307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.408997 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.409009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.411486 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.411494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.411759 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.411914 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.412052 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.413294 (+ 1242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14618,"mutex_wait_us":1149,"thread_start_us":15611,"threads_started":1}
I20260807 18:09:24.414366 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.383859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40332 (local address 127.8.62.131:37355)
0807 18:09:24.402470 (+ 18611us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.402479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.402506 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:09:24.402610 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.402618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.402888 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.403116 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:24.404229 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.406291 (+ 2062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.409278 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.410807 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.410942 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.411196 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.413996 (+ 2800us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.414135 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18412,"thread_start_us":18261,"threads_started":1}
I20260807 18:09:24.415422 23519 tablet_service.cc:1467] Processing CreateTablet for tablet 9527d74f69ee41c8be04f2162d263865 (DEFAULT_TABLE table=test-table [id=8a1dc909877f44739fb2c1fc58e3c643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:24.416860 23519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9527d74f69ee41c8be04f2162d263865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:24.417729 23798 tablet_service.cc:1467] Processing CreateTablet for tablet 9527d74f69ee41c8be04f2162d263865 (DEFAULT_TABLE table=test-table [id=8a1dc909877f44739fb2c1fc58e3c643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:24.417729 23662 tablet_service.cc:1467] Processing CreateTablet for tablet 9527d74f69ee41c8be04f2162d263865 (DEFAULT_TABLE table=test-table [id=8a1dc909877f44739fb2c1fc58e3c643]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:24.419171 23662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9527d74f69ee41c8be04f2162d263865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:24.419167 23798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9527d74f69ee41c8be04f2162d263865. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:24.426352 23882 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Bootstrap starting.
I20260807 18:09:24.430367 23882 tablet_bootstrap.cc:654] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:24.431878 23882 log.cc:824] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:24.434906 23882 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: No bootstrap required, opened a new log
I20260807 18:09:24.435228 23882 ts_tablet_manager.cc:1397] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 18:09:24.436466 23885 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Bootstrap starting.
I20260807 18:09:24.437390 23884 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Bootstrap starting.
I20260807 18:09:24.441502 23884 tablet_bootstrap.cc:654] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:24.442919 23884 log.cc:824] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:24.443811 23885 tablet_bootstrap.cc:654] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:24.445019 23885 log.cc:824] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:24.447304 23882 raft_consensus.cc:348] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.447995 23882 raft_consensus.cc:374] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:24.448143 23882 raft_consensus.cc:729] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad6aed6744d4ec7a2fb8e41fadb9b29, State: Initialized, Role: FOLLOWER
I20260807 18:09:24.448590 23885 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: No bootstrap required, opened a new log
I20260807 18:09:24.449016 23885 ts_tablet_manager.cc:1397] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.005s
I20260807 18:09:24.449057 23882 consensus_queue.cc:260] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.451195 23882 ts_tablet_manager.cc:1428] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
I20260807 18:09:24.452689 23884 tablet_bootstrap.cc:492] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: No bootstrap required, opened a new log
I20260807 18:09:24.452996 23884 ts_tablet_manager.cc:1397] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
I20260807 18:09:24.460335 23885 raft_consensus.cc:348] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.460467 23884 raft_consensus.cc:348] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.460860 23885 raft_consensus.cc:374] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:24.460965 23885 raft_consensus.cc:729] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b0d55dba8b2447e990c5438c4299112, State: Initialized, Role: FOLLOWER
I20260807 18:09:24.460970 23884 raft_consensus.cc:374] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:24.461093 23884 raft_consensus.cc:729] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02954b36666e4e8dab6cdc5dc97a6b4a, State: Initialized, Role: FOLLOWER
I20260807 18:09:24.461678 23885 consensus_queue.cc:260] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.461716 23884 consensus_queue.cc:260] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.462967 23731 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
I20260807 18:09:24.463724 23884 ts_tablet_manager.cc:1428] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260807 18:09:24.463752 23885 ts_tablet_manager.cc:1428] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Time spent starting tablet: real 0.015s user 0.007s sys 0.005s
I20260807 18:09:24.464668 23866 heartbeater.cc:503] Master 127.8.62.190:46687 was elected leader, sending a full tablet report...
I20260807 18:09:24.500164 23889 raft_consensus.cc:482] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:09:24.500519 23889 raft_consensus.cc:504] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.502394 23889 leader_election.cc:288] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987), 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355)
I20260807 18:09:24.509069 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.502887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:50291 (local address 127.8.62.129:32987)
0807 18:09:24.503138 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.503147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.503786 (+ 639us) server_negotiation.cc:407] Connection header received
0807 18:09:24.504028 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.504035 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.504103 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.504203 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:24.504783 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.505538 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.507286 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.508093 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.508284 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.508363 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.508674 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.508812 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:09:24.509307 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.502817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:32987 (local address 127.8.62.130:50291)
0807 18:09:24.503664 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.503682 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.503868 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.504219 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.504224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.504284 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:24.504617 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.504626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.505670 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.505678 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.507124 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.507136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.508320 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.508326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.508471 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.508495 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.508647 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.508776 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"spinlock_wait_cycles":31104,"thread_start_us":887,"threads_started":1}
I20260807 18:09:24.510375 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.502865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:37355 (local address 127.8.62.130:51325)
0807 18:09:24.504088 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:24.504104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:24.504290 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:24.504715 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:24.504720 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:24.504746 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:24.505150 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.505159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.506254 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.506261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.507778 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:24.507791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.509387 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.509399 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:24.509626 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.509672 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 18:09:24.509788 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:09:24.509925 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"spinlock_wait_cycles":1280,"thread_start_us":458,"threads_started":1}
I20260807 18:09:24.510388 23540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9527d74f69ee41c8be04f2162d263865" candidate_uuid: "0b0d55dba8b2447e990c5438c4299112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" is_pre_election: true
I20260807 18:09:24.510231 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.503211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:51325 (local address 127.8.62.131:37355)
0807 18:09:24.503475 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.503482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.504218 (+ 736us) server_negotiation.cc:407] Connection header received
0807 18:09:24.504461 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.504465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.504537 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.504698 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:24.505319 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.506091 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.507954 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.508902 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.509113 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.509204 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.509819 (+ 615us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.509956 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:09:24.511036 23540 raft_consensus.cc:2454] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b0d55dba8b2447e990c5438c4299112 in term 0.
I20260807 18:09:24.511631 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9527d74f69ee41c8be04f2162d263865" candidate_uuid: "0b0d55dba8b2447e990c5438c4299112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" is_pre_election: true
I20260807 18:09:24.511729 23611 leader_election.cc:302] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0d55dba8b2447e990c5438c4299112, 5ad6aed6744d4ec7a2fb8e41fadb9b29; no voters:
I20260807 18:09:24.512138 23889 raft_consensus.cc:2781] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:09:24.512121 23820 raft_consensus.cc:2454] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b0d55dba8b2447e990c5438c4299112 in term 0.
I20260807 18:09:24.512254 23889 raft_consensus.cc:482] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:09:24.512367 23889 raft_consensus.cc:3037] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:24.514721 23889 raft_consensus.cc:504] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.515443 23889 leader_election.cc:288] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [CANDIDATE]: Term 1 election: Requested vote from peers 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987), 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355)
I20260807 18:09:24.516009 23540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9527d74f69ee41c8be04f2162d263865" candidate_uuid: "0b0d55dba8b2447e990c5438c4299112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
I20260807 18:09:24.516055 23820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9527d74f69ee41c8be04f2162d263865" candidate_uuid: "0b0d55dba8b2447e990c5438c4299112" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
I20260807 18:09:24.516237 23540 raft_consensus.cc:3037] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:24.516280 23820 raft_consensus.cc:3037] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:24.518561 23540 raft_consensus.cc:2454] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b0d55dba8b2447e990c5438c4299112 in term 1.
I20260807 18:09:24.518851 23820 raft_consensus.cc:2454] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b0d55dba8b2447e990c5438c4299112 in term 1.
I20260807 18:09:24.519059 23611 leader_election.cc:302] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b0d55dba8b2447e990c5438c4299112, 5ad6aed6744d4ec7a2fb8e41fadb9b29; no voters:
I20260807 18:09:24.519459 23889 raft_consensus.cc:2781] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:24.519966 23889 raft_consensus.cc:686] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 1 LEADER]: Becoming Leader. State: Replica: 0b0d55dba8b2447e990c5438c4299112, State: Running, Role: LEADER
I20260807 18:09:24.520576 23889 consensus_queue.cc:237] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } }
I20260807 18:09:24.525668 23387 catalog_manager.cc:5495] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "0b0d55dba8b2447e990c5438c4299112" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b0d55dba8b2447e990c5438c4299112" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 } health_report { overall_health: UNKNOWN } } }
W20260807 18:09:24.558424 23590 tablet.cc:2369] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:09:24.565026 23732 tablet.cc:2369] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:09:24.568490 23867 tablet.cc:2369] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:24.570035 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.563733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.8.62.130:40531)
0807 18:09:24.563929 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.563934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.563955 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:09:24.564058 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.564062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.564153 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.564291 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:09:24.564684 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.566018 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.567553 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.568057 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.568305 (+ 248us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.568866 (+ 561us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:09:24.569402 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:09:24.569572 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.569659 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.569744 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:09:24.579193 23820 raft_consensus.cc:1261] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Refusing update from remote peer 0b0d55dba8b2447e990c5438c4299112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:24.579180 23540 raft_consensus.cc:1261] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Refusing update from remote peer 0b0d55dba8b2447e990c5438c4299112: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:24.580351 23889 consensus_queue.cc:1035] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02954b36666e4e8dab6cdc5dc97a6b4a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 37355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:24.580856 23893 consensus_queue.cc:1035] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 32987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:24.592485 23896 mvcc.cc:205] Tried to move back new op lower bound from 7315972770103914496 to 7315972769880719360. Current Snapshot: MvccSnapshot[applied={T|T < 7315972770103914496}]
I20260807 18:09:24.606640 23387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:46194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:24.606927 23387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:46194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:24.610945 23387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Sending DeleteTablet for 3 replicas of tablet 9527d74f69ee41c8be04f2162d263865
I20260807 18:09:24.612151 23662 tablet_service.cc:1514] Processing DeleteTablet for tablet 9527d74f69ee41c8be04f2162d263865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:24 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42318
I20260807 18:09:24.612174 23519 tablet_service.cc:1514] Processing DeleteTablet for tablet 9527d74f69ee41c8be04f2162d263865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:24 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:55526
I20260807 18:09:24.612629 23798 tablet_service.cc:1514] Processing DeleteTablet for tablet 9527d74f69ee41c8be04f2162d263865 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:24 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40332
I20260807 18:09:24.613337 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23428
I20260807 18:09:24.625214 23905 tablet_replica.cc:330] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: stopping tablet replica
I20260807 18:09:24.625913 23906 tablet_replica.cc:330] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: stopping tablet replica
I20260807 18:09:24.626595 23906 raft_consensus.cc:2229] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:09:24.627241 23905 raft_consensus.cc:2229] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:24.628353 23905 raft_consensus.cc:2258] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:24.628827 23906 raft_consensus.cc:2258] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:24.630065 23907 tablet_replica.cc:330] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: stopping tablet replica
I20260807 18:09:24.630580 23907 raft_consensus.cc:2229] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:24.631198 23907 raft_consensus.cc:2258] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:24.631820 23905 ts_tablet_manager.cc:1905] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:24.634400 23907 ts_tablet_manager.cc:1905] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:24.635910 23906 ts_tablet_manager.cc:1905] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:24.639026 23905 ts_tablet_manager.cc:1918] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:24.639182 23905 log.cc:1196] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/9527d74f69ee41c8be04f2162d263865
I20260807 18:09:24.639671 23905 ts_tablet_manager.cc:1939] T 9527d74f69ee41c8be04f2162d263865 P 5ad6aed6744d4ec7a2fb8e41fadb9b29: Deleting consensus metadata
I20260807 18:09:24.641988 23907 ts_tablet_manager.cc:1918] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:24.642246 23907 log.cc:1196] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/9527d74f69ee41c8be04f2162d263865
I20260807 18:09:24.642905 23907 ts_tablet_manager.cc:1939] T 9527d74f69ee41c8be04f2162d263865 P 02954b36666e4e8dab6cdc5dc97a6b4a: Deleting consensus metadata
I20260807 18:09:24.643322 23371 catalog_manager.cc:4841] TS 5ad6aed6744d4ec7a2fb8e41fadb9b29 (127.8.62.129:32987): tablet 9527d74f69ee41c8be04f2162d263865 (table test-table [id=8a1dc909877f44739fb2c1fc58e3c643]) successfully deleted
I20260807 18:09:24.645053 23880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.614286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.8.62.129:32987)
0807 18:09:24.614834 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.614843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.614867 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:09:24.614937 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.614945 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.615077 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.615209 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:24.617676 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.619273 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.622961 (+ 3688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.624018 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.624197 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.629583 (+ 5386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:24.629599 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:24.629622 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:24.629673 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:24.634357 (+ 4684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.641611 (+ 7254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.641623 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.641626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.641843 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.642315 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.642320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.642324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.642823 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:24.643175 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.644340 (+ 1165us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.644615 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375}
I20260807 18:09:24.645438 23374 catalog_manager.cc:4841] TS 02954b36666e4e8dab6cdc5dc97a6b4a (127.8.62.131:37355): tablet 9527d74f69ee41c8be04f2162d263865 (table test-table [id=8a1dc909877f44739fb2c1fc58e3c643]) successfully deleted
I20260807 18:09:24.645906 23584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:24.646833 23906 ts_tablet_manager.cc:1918] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:24.647032 23906 log.cc:1196] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/9527d74f69ee41c8be04f2162d263865
I20260807 18:09:24.647730 23906 ts_tablet_manager.cc:1939] T 9527d74f69ee41c8be04f2162d263865 P 0b0d55dba8b2447e990c5438c4299112: Deleting consensus metadata
I20260807 18:09:24.649602 23372 catalog_manager.cc:4841] TS 0b0d55dba8b2447e990c5438c4299112 (127.8.62.130:40531): tablet 9527d74f69ee41c8be04f2162d263865 (table test-table [id=8a1dc909877f44739fb2c1fc58e3c643]) successfully deleted
I20260807 18:09:24.750527 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23428
I20260807 18:09:24.770605 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23593
I20260807 18:09:24.789228 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.771432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.8.62.130:40531)
0807 18:09:24.771913 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.771917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.771936 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:09:24.771988 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.771993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.772052 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.772177 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:24.773664 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.774482 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.776009 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.776516 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.776832 (+ 316us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.781218 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:24.781239 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:24.781262 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:24.781321 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:24.785695 (+ 4374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.786712 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.786720 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.786722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.786889 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.787330 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.787333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.787335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.787789 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:24.788056 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.788404 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.788811 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357}
I20260807 18:09:24.790184 23726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:24.910878 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23593
I20260807 18:09:24.932718 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23734
I20260807 18:09:24.946900 23879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:24.933524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40344 (local address 127.8.62.131:37355)
0807 18:09:24.933759 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:24.933764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:24.933784 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:09:24.933843 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:24.933847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:24.933908 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:24.934004 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:24.935358 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.936148 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.937651 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:24.938069 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:24.938171 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:24.941088 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:24.941105 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:24.941121 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:24.941159 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:24.943925 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.944693 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.944700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.944703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.944847 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:24.945272 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:24.945276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:24.945278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:24.945667 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:24.945913 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:24.946320 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 18:09:24.946562 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 18:09:24.947746 23861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:25.041688 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23734
I20260807 18:09:25.061303 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23352
I20260807 18:09:25.062539 23421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:25.180708 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23352
I20260807 18:09:25.206192 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--webserver_interface=127.8.62.190
--webserver_port=41707
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:25.572412 23912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:25.572849 23912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:25.572968 23912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:25.580861 23912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:25.580996 23912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:25.581044 23912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:25.581104 23912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:25.581149 23912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:25.591483 23912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:25.593099 23912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:25.594549 23912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:25.607615 23917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:25.607621 23920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:25.607686 23918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:25.608592 23912 server_base.cc:1034] running on GCE node
I20260807 18:09:25.609462 23912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:25.611114 23912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:25.612344 23912 hybrid_clock.cc:648] HybridClock initialized: now 1786126165612281 us; error 54 us; skew 500 ppm
I20260807 18:09:25.612821 23912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:25 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126165, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:25.616457 23912 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:25.618180 23912 webserver.cc:460] Webserver started at http://127.8.62.190:41707/ using document root <none> and password file <none>
I20260807 18:09:25.618956 23912 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:25.619117 23912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:25.625463 23912 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:09:25.628837 23927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:25.630625 23912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:09:25.630868 23912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "9b7e4e12855c4a018c3a32d3e284b9cb"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "a5017e4e83223f60850c6f51b522980e"
server_key_iv: "298618ffb9db815e15b6c7074ffb17b3"
server_key_version: "encryptionkey@0"
I20260807 18:09:25.631997 23912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:25.655889 23912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:25.660360 23912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:25.660677 23912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:25.680882 23912 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46687
I20260807 18:09:25.680907 23981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46687 every 8 connection(s)
I20260807 18:09:25.682727 23912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:25.684374 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23912
I20260807 18:09:25.685606 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:32987
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=39943
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:25.694782 23982 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:09:25.699275 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap starting.
I20260807 18:09:25.723040 23982 log.cc:824] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:25.741081 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:09:25.741730 23982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb: Bootstrap complete.
I20260807 18:09:25.751674 23982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:25.752751 23982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Initialized, Role: FOLLOWER
I20260807 18:09:25.753508 23982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:25.753798 23982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:25.753948 23982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:25.754138 23982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:09:25.758548 23982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:25.759238 23982 leader_election.cc:302] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b7e4e12855c4a018c3a32d3e284b9cb; no voters:
I20260807 18:09:25.760079 23982 leader_election.cc:288] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:09:25.760254 23986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:09:25.761538 23986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [term 3 LEADER]: Becoming Leader. State: Replica: 9b7e4e12855c4a018c3a32d3e284b9cb, State: Running, Role: LEADER
I20260807 18:09:25.762171 23986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } }
I20260807 18:09:25.762639 23982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:25.765108 23987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:25.765595 23987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:25.769529 23988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b7e4e12855c4a018c3a32d3e284b9cb. Latest consensus state: current_term: 3 leader_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b7e4e12855c4a018c3a32d3e284b9cb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46687 } } }
I20260807 18:09:25.769940 23988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b7e4e12855c4a018c3a32d3e284b9cb [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:25.771893 23992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:25.784446 23992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:25.786690 23992 catalog_manager.cc:1259] Loaded cluster ID: 567ce33278c64485afd6caf944c566bc
I20260807 18:09:25.786864 23992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 18:09:25.790154 23992 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fd9ad1789ad google::LogMessage::Fail() at ??:0
@ 0x7fd9ad17b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fd9ad1784de google::LogMessage::Flush() at ??:0
@ 0x7fd9ad179a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fd9c2bab512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fd9c2ba8976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fd9c2c30610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fd9c2c30439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd9afdc4345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fd9afdc791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fd9afdc7799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fd9afda2eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fd9b307c6db start_thread at ??:0
@ 0x7fd9ac01261f clone at ??:0
W20260807 18:09:26.084784 23984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:26.085147 23984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:26.085323 23984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:26.093472 23984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:26.093660 23984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:26.093844 23984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:26.104710 23984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:32987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=39943
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:26.106200 23984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:26.107682 23984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:26.122005 24030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:26.122133 24031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:26.122150 24033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:26.123090 23984 server_base.cc:1034] running on GCE node
I20260807 18:09:26.123806 23984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:26.124989 23984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:26.126273 23984 hybrid_clock.cc:648] HybridClock initialized: now 1786126166126206 us; error 47 us; skew 500 ppm
I20260807 18:09:26.126735 23984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:26 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:26.130676 23984 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:26.132524 23984 webserver.cc:460] Webserver started at http://127.8.62.129:39943/ using document root <none> and password file <none>
I20260807 18:09:26.133417 23984 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:26.133697 23984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:26.140902 23984 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.002s
I20260807 18:09:26.144760 24040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.146399 23984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:26.146636 23984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "5ad6aed6744d4ec7a2fb8e41fadb9b29"
format_stamp: "Formatted at 2026-08-07 18:09:19 on dist-test-slave-rnrw"
server_key: "83050f7723e90e60d271b8d40b81a0a6"
server_key_iv: "8ad0b85a145dbd8de483812bae0c036c"
server_key_version: "encryptionkey@0"
I20260807 18:09:26.147841 23984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:26.175565 23984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:26.181140 23984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:26.181476 23984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:26.182679 23984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:26.184331 23984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:26.184484 23984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.184655 23984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:26.184751 23984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.224428 23984 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:32987
I20260807 18:09:26.224529 24156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:32987 every 8 connection(s)
I20260807 18:09:26.226136 23984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:26.228804 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 23984
I20260807 18:09:26.229494 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:40531
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=42911
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:09:26.232044 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.229680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:39473)
0807 18:09:26.230674 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.231002 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":424,"threads_started":1}
W20260807 18:09:26.232867 24157 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:46687 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:46687: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
I20260807 18:09:26.233990 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.233579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:35385)
0807 18:09:26.233790 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.233866 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:09:26.235013 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.234654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:56729)
0807 18:09:26.234808 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.234896 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:09:26.235332 24157 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:09:26.597855 24160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:26.598213 24160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:26.598389 24160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:26.606297 24160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:26.606436 24160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:26.606629 24160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:26.617129 24160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:40531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=42911
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:26.618595 24160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:26.620064 24160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:26.634995 24166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:26.635125 24165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:26.635151 24168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:26.636066 24160 server_base.cc:1034] running on GCE node
I20260807 18:09:26.637077 24160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:26.638142 24160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:26.639370 24160 hybrid_clock.cc:648] HybridClock initialized: now 1786126166639320 us; error 37 us; skew 500 ppm
I20260807 18:09:26.639803 24160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:26 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126166, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:26.643494 24160 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:26.645571 24160 webserver.cc:460] Webserver started at http://127.8.62.130:42911/ using document root <none> and password file <none>
I20260807 18:09:26.646394 24160 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:26.646561 24160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:26.652933 24160 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:09:26.655994 24175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.657665 24160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:09:26.657878 24160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "0b0d55dba8b2447e990c5438c4299112"
format_stamp: "Formatted at 2026-08-07 18:09:20 on dist-test-slave-rnrw"
server_key: "6487df446f390e68873101430f929f6a"
server_key_iv: "b626f99def3b97eb0424075868e38696"
server_key_version: "encryptionkey@0"
I20260807 18:09:26.659439 24160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:26.686509 24160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:26.691636 24160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:26.691906 24160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:26.692988 24160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:26.694422 24160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:26.694527 24160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.694624 24160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:26.694684 24160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:26.731575 24160 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:40531
I20260807 18:09:26.731676 24294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:40531 every 8 connection(s)
I20260807 18:09:26.733309 24160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:26.738873 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24160
I20260807 18:09:26.739135 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.736739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:43271)
0807 18:09:26.737626 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.737875 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260807 18:09:26.739684 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:37355
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:46687
--webserver_port=42003
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:26.740060 24295 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:46687 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:46687: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
I20260807 18:09:26.741200 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.740775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:42861)
0807 18:09:26.740958 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.741071 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:09:26.742251 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:26.741876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:49737)
0807 18:09:26.742034 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:26.742128 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:09:26.742544 24295 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:09:27.095825 24298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:27.096227 24298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:27.096410 24298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:27.105382 24298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:27.105561 24298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:27.105757 24298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:27.115969 24298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:37355
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=42003
--tserver_master_addrs=127.8.62.190:46687
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:27.117394 24298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:27.118794 24298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:27.132985 24304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:27.133013 24303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:27.133076 24306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:27.134080 24298 server_base.cc:1034] running on GCE node
I20260807 18:09:27.134845 24298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:27.135991 24298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:27.137243 24298 hybrid_clock.cc:648] HybridClock initialized: now 1786126167137187 us; error 50 us; skew 500 ppm
I20260807 18:09:27.137663 24298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:27 dist-test-slave-rnrw krb5kdc[22773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126167, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:27.141250 24298 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:27.142990 24298 webserver.cc:460] Webserver started at http://127.8.62.131:42003/ using document root <none> and password file <none>
I20260807 18:09:27.143874 24298 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:27.144034 24298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:27.150496 24298 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:09:27.154040 24314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:27.155984 24298 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 18:09:27.156242 24298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "02954b36666e4e8dab6cdc5dc97a6b4a"
format_stamp: "Formatted at 2026-08-07 18:09:21 on dist-test-slave-rnrw"
server_key: "1d121f3e61e2f09f0bdafc9ae985b0c7"
server_key_iv: "1d6f39163906be929afb910d6a526a57"
server_key_version: "encryptionkey@0"
I20260807 18:09:27.157583 24298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:27.175463 24298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:27.180981 24298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:27.181329 24298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:27.182518 24298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:27.184051 24298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:27.184185 24298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:27.184324 24298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:27.184410 24298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:27.222401 24298 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:37355
I20260807 18:09:27.222498 24432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:37355 every 8 connection(s)
I20260807 18:09:27.224009 24298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:27.227355 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24298
I20260807 18:09:27.230557 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:27.227477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:43411)
0807 18:09:27.228596 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:27.228965 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":428,"threads_started":1}
W20260807 18:09:27.231554 24433 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:46687 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:46687: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
I20260807 18:09:27.232760 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:27.232179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:51737)
0807 18:09:27.232433 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:27.232537 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 18:09:27.234413 24435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:27.233933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:60545)
0807 18:09:27.234145 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:27.234249 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 18:09:27.234887 24433 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:09:27.237012 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:27.235992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:49257)
0807 18:09:27.236639 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:27.236751 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":269,"threads_started":1}
I20260807 18:09:27.744383 24438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:27.743282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:33301)
0807 18:09:27.744012 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:27.744118 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":426,"threads_started":1}
I20260807 18:09:28.236529 24439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:28.235609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:56373)
0807 18:09:28.236153 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:28.236267 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":269,"threads_started":1}
I20260807 18:09:28.238977 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:28.238038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:56781)
0807 18:09:28.238677 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:28.238784 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":259,"threads_started":1}
I20260807 18:09:28.746383 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:28.745572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:38365)
0807 18:09:28.746117 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:28.746207 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":265,"threads_started":1}
I20260807 18:09:29.238513 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:29.237735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:47975)
0807 18:09:29.238242 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:29.238330 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":268,"threads_started":1}
I20260807 18:09:29.241042 24443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:29.240028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:33401)
0807 18:09:29.240693 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:29.240785 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":371,"threads_started":1}
I20260807 18:09:29.748337 24445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:29.747521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:54843)
0807 18:09:29.748018 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:29.748136 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":253,"threads_started":1}
I20260807 18:09:30.240478 24446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:30.239642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:37107)
0807 18:09:30.240176 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:30.240272 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":274,"threads_started":1}
I20260807 18:09:30.243072 24447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:30.242177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:57547)
0807 18:09:30.242744 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:30.242833 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":283,"threads_started":1}
I20260807 18:09:30.750471 24448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:30.749575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:41295)
0807 18:09:30.750169 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:30.750273 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":280,"threads_started":1}
I20260807 18:09:31.242430 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:31.241637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:36883)
0807 18:09:31.242151 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:31.242242 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":261,"threads_started":1}
I20260807 18:09:31.245131 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:31.244209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:38021)
0807 18:09:31.244795 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:31.244887 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":277,"threads_started":1}
I20260807 18:09:31.752576 24451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:31.751688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:53377)
0807 18:09:31.752209 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:31.752303 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":246,"threads_started":1}
I20260807 18:09:32.244419 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:32.243574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:43567)
0807 18:09:32.244152 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:32.244241 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":276,"threads_started":1}
I20260807 18:09:32.246855 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:32.246155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:60427)
0807 18:09:32.246595 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:32.246691 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":233,"threads_started":1}
I20260807 18:09:32.754529 24454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:32.753711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:36561)
0807 18:09:32.754219 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:32.754323 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":256,"threads_started":1}
I20260807 18:09:33.246270 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:33.245404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:52305)
0807 18:09:33.245987 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:33.246095 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":344,"threads_started":1}
I20260807 18:09:33.249037 24456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:33.247798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:59315)
0807 18:09:33.248635 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:33.248748 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":436,"threads_started":1}
I20260807 18:09:33.756460 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:33.755618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:49851)
0807 18:09:33.756142 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:33.756226 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":221,"threads_started":1}
I20260807 18:09:34.248348 24458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:34.247472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:47683)
0807 18:09:34.248031 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:34.248134 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":233,"threads_started":1}
I20260807 18:09:34.250849 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:34.250039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:37747)
0807 18:09:34.250591 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:34.250675 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":312,"threads_started":1}
I20260807 18:09:34.758548 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:34.757713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:46733)
0807 18:09:34.758276 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:34.758371 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":304,"threads_started":1}
I20260807 18:09:35.250432 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:35.249570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:48771)
0807 18:09:35.250169 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:35.250258 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":348,"threads_started":1}
I20260807 18:09:35.252877 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:35.251906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:44651)
0807 18:09:35.252594 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:35.252708 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":320,"threads_started":1}
I20260807 18:09:35.760703 24463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:35.759699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:40001)
0807 18:09:35.760240 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:35.760346 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":275,"threads_started":1}
I20260807 18:09:36.252655 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:36.251555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:35403)
0807 18:09:36.252267 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:36.252379 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":326,"threads_started":1}
I20260807 18:09:36.254844 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:36.253933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:36105)
0807 18:09:36.254526 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:36.254661 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":333,"threads_started":1}
I20260807 18:09:36.762497 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:36.761729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:46505)
0807 18:09:36.762236 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:36.762326 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":289,"threads_started":1}
I20260807 18:09:37.254597 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:37.253766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:35399)
0807 18:09:37.254335 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:37.254416 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":340,"threads_started":1}
I20260807 18:09:37.256515 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:37.255739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:49801)
0807 18:09:37.256228 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:37.256330 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":234,"threads_started":1}
I20260807 18:09:37.764521 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:37.763643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:52013)
0807 18:09:37.764161 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:37.764264 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":240,"threads_started":1}
I20260807 18:09:38.256801 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:38.255748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:38493)
0807 18:09:38.256367 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:38.256508 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":344,"threads_started":1}
I20260807 18:09:38.258301 24471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:38.257533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:47243)
0807 18:09:38.258051 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:38.258145 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":284,"threads_started":1}
I20260807 18:09:38.766419 24472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:38.765649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:56377)
0807 18:09:38.766155 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:38.766251 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":252,"threads_started":1}
I20260807 18:09:39.258666 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:39.257832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:60347)
0807 18:09:39.258416 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:39.258500 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":350,"threads_started":1}
I20260807 18:09:39.260140 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:39.259270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:59423)
0807 18:09:39.259819 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:39.259922 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":284,"threads_started":1}
I20260807 18:09:39.768750 24476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:39.767753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:33491)
0807 18:09:39.768338 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:39.768464 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":288,"threads_started":1}
I20260807 18:09:40.260790 24477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:40.259739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:33109)
0807 18:09:40.260425 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:40.260531 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":353,"threads_started":1}
I20260807 18:09:40.261955 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:40.261178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:59793)
0807 18:09:40.261681 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:40.261772 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":233,"threads_started":1}
I20260807 18:09:40.770859 24480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:40.769990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:41789)
0807 18:09:40.770565 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:40.770662 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":260,"threads_started":1}
I20260807 18:09:41.262751 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:41.261897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:53533)
0807 18:09:41.262457 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:41.262561 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":242,"threads_started":1}
I20260807 18:09:41.264040 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:41.263148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:58157)
0807 18:09:41.263739 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:41.263863 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":270,"threads_started":1}
I20260807 18:09:41.773073 24483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:41.772104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.130:53297)
0807 18:09:41.772728 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:41.772857 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":268,"threads_started":1}
I20260807 18:09:42.239114 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 23984
I20260807 18:09:42.265069 24485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:42.264125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.131:37565)
0807 18:09:42.264669 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:42.264787 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":275,"threads_started":1}
I20260807 18:09:42.266376 24486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:42.265144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46687 (local address 127.8.62.129:40531)
0807 18:09:42.265805 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:42.266031 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:46687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":349,"threads_started":1}
I20260807 18:09:42.268190 24484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:42.240938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43946 (local address 127.8.62.129:32987)
0807 18:09:42.241762 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:42.241787 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:42.241838 (+ 51us) server_negotiation.cc:407] Connection header received
0807 18:09:42.244092 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:42.244105 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:42.244699 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:42.245079 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:42.247348 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.248884 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.250340 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.251141 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.251297 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:42.259802 (+ 8505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:42.259813 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:42.259833 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:42.259888 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:42.263595 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.264523 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.264535 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.264565 (+ 30us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.264848 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.265386 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.265394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.265400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.266384 (+ 984us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:42.266737 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:42.267319 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 18:09:42.267889 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":412,"threads_started":1}
I20260807 18:09:42.269757 24151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:42.360271 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 23984
I20260807 18:09:42.380633 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24160
I20260807 18:09:42.402177 24488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:42.381916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.8.62.130:40531)
0807 18:09:42.382747 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:42.382762 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:42.382803 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:09:42.384434 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:42.384445 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:42.385211 (+ 766us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:42.385703 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:42.387416 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.388852 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.390255 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.391002 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.391224 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:42.394402 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:42.394411 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:42.394429 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:42.394481 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:42.398316 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.399177 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.399185 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.399193 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.399396 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.399850 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.399856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.399859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.400469 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:42.400850 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:42.401350 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:09:42.401864 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":309,"threads_started":1}
I20260807 18:09:42.403805 24290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:42.493072 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24160
I20260807 18:09:42.512856 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24298
I20260807 18:09:42.534206 24490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:42.513988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51668 (local address 127.8.62.131:37355)
0807 18:09:42.514633 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:42.514649 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:42.514704 (+ 55us) server_negotiation.cc:407] Connection header received
0807 18:09:42.516951 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:42.516963 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:42.517491 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:42.517917 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:42.519781 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.521282 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.522751 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:42.523513 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:42.523741 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:42.527162 (+ 3421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:42.527171 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:42.527191 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:42.527236 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:42.530316 (+ 3080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.531117 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.531124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.531131 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.531308 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:42.531735 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:42.531740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:42.531745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:42.532342 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:42.532958 (+ 616us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:42.533338 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 18:09:42.533825 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":309,"threads_started":1}
I20260807 18:09:42.535650 24427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:42.620046 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24298
2026-08-07T18:09:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25184 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:09:42 dist-test-slave-rnrw krb5kdc[24494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:09:42 dist-test-slave-rnrw krb5kdc[24494](info): set up 2 sockets
Aug 07 18:09:42 dist-test-slave-rnrw krb5kdc[24494](info): commencing operation
krb5kdc: starting...
W20260807 18:09:43.804023 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:09:43 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:09:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:09:44.078321 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:43875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:44.447619 24507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:44.447949 24507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:44.448120 24507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:44.456023 24507 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:09:44.456136 24507 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:09:44.456175 24507 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:09:44.456279 24507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:09:44.456411 24507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:44.456525 24507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:44.456638 24507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:09:44.456723 24507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:09:44.467156 24507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:43875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:44.468706 24507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:44.470113 24507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:44.483390 24517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:44.483299 24520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:44.483353 24518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:44.483925 24507 server_base.cc:1034] running on GCE node
I20260807 18:09:44.485543 24507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:44.487627 24507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:44.488910 24507 hybrid_clock.cc:648] HybridClock initialized: now 1786126184488831 us; error 63 us; skew 500 ppm
I20260807 18:09:44.489243 24507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:44 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126184, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:44.492717 24507 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:09:44.494283 24507 webserver.cc:460] Webserver started at http://127.8.62.190:41415/ using document root <none> and password file <none>
I20260807 18:09:44.495007 24507 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:44.495103 24507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:44.495371 24507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:44.498564 24507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "55b5965c6aae4f5f8bb6ef8369b22d75"
format_stamp: "Formatted at 2026-08-07 18:09:44 on dist-test-slave-rnrw"
server_key: "176f5f215a280f85ae29c469a2b8f7fd"
server_key_iv: "3339d3e80c31ec2e09fad16d1993eae9"
server_key_version: "encryptionkey@0"
I20260807 18:09:44.499321 24507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "55b5965c6aae4f5f8bb6ef8369b22d75"
format_stamp: "Formatted at 2026-08-07 18:09:44 on dist-test-slave-rnrw"
server_key: "176f5f215a280f85ae29c469a2b8f7fd"
server_key_iv: "3339d3e80c31ec2e09fad16d1993eae9"
server_key_version: "encryptionkey@0"
I20260807 18:09:44.504953 24507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:09:44.508814 24527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:44.510361 24507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:09:44.510577 24507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "55b5965c6aae4f5f8bb6ef8369b22d75"
format_stamp: "Formatted at 2026-08-07 18:09:44 on dist-test-slave-rnrw"
server_key: "176f5f215a280f85ae29c469a2b8f7fd"
server_key_iv: "3339d3e80c31ec2e09fad16d1993eae9"
server_key_version: "encryptionkey@0"
I20260807 18:09:44.510833 24507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:44.531018 24507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:44.536352 24507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:44.536762 24507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:44.558642 24507 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:43875
I20260807 18:09:44.558674 24581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:43875 every 8 connection(s)
I20260807 18:09:44.560125 24507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:09:44.564412 24582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:44.567368 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24507
I20260807 18:09:44.567621 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:09:44.568303 8442 external_mini_cluster.cc:1428] Setting key 3d45750b700225af8403ee438892ddd7
I20260807 18:09:44.578024 24582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap starting.
I20260807 18:09:44.582665 24582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:44.584297 24582 log.cc:824] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:09:44 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:44.587913 24582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: No bootstrap required, opened a new log
I20260807 18:09:44.597764 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:44.570573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57038 (local address 127.8.62.190:43875)
0807 18:09:44.571420 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:44.571432 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:44.571476 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:09:44.573346 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:44.573370 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:44.573822 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:44.574183 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:44.576312 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:44.577939 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:44.580135 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:44.581145 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:44.581302 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:44.587258 (+ 5956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:44.587267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:44.587284 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:44.587329 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:44.591868 (+ 4539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:44.592967 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:44.592978 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:44.592989 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:44.593259 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:44.593794 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:44.593800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:44.593803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:44.594726 (+ 923us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:44.595218 (+ 492us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:44.595607 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 18:09:44.596186 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":392,"threads_started":1}
I20260807 18:09:44.599732 24582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:09:44.600121 24582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:44.600260 24582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Initialized, Role: FOLLOWER
I20260807 18:09:44.601025 24582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:09:44.601284 24582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:09:44.601449 24582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:09:44.601646 24582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:44.603921 24582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:09:44.604485 24582 leader_election.cc:302] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b5965c6aae4f5f8bb6ef8369b22d75; no voters:
I20260807 18:09:44.605196 24582 leader_election.cc:288] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:09:44.605348 24587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:44.606094 24587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 LEADER]: Becoming Leader. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Running, Role: LEADER
I20260807 18:09:44.606716 24587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:09:44.607679 24582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:09:44.610250 24589 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b5965c6aae4f5f8bb6ef8369b22d75. Latest consensus state: current_term: 1 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:09:44.610910 24589 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:44.610707 24588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:09:44.611126 24588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:09:44.614980 24594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:09:44.620909 24594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:09:44.631028 24594 catalog_manager.cc:1347] Generated new cluster ID: 8dcdf638728c4bc29386f78203d99707
I20260807 18:09:44.631191 24594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:09:44.660611 24594 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:09:44.662650 24594 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:09:44.669044 24594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Generated new TSK 0
I20260807 18:09:44.669951 24594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:09:44.741266 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:45.097606 24609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:45.097918 24609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:45.098081 24609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:45.105926 24609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:45.106053 24609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:45.106222 24609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:09:45.117200 24609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:45.118597 24609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:45.119967 24609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:45.134796 24614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:45.134768 24615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:45.134985 24618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:45.135799 24609 server_base.cc:1034] running on GCE node
I20260807 18:09:45.136641 24609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:45.137691 24609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:45.138927 24609 hybrid_clock.cc:648] HybridClock initialized: now 1786126185138886 us; error 39 us; skew 500 ppm
I20260807 18:09:45.139254 24609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:45 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:45.142903 24609 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:09:45.144590 24609 webserver.cc:460] Webserver started at http://127.8.62.129:37125/ using document root <none> and password file <none>
I20260807 18:09:45.145387 24609 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:45.145551 24609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:45.145905 24609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:45.149235 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "acfbdf31457c08292df1e27a7607bbcd"
server_key_iv: "6220de5fa1e508e74edf781a6a2e91cf"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.150087 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "acfbdf31457c08292df1e27a7607bbcd"
server_key_iv: "6220de5fa1e508e74edf781a6a2e91cf"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.156101 24609 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 18:09:45.160354 24625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.162014 24609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:09:45.162243 24609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "acfbdf31457c08292df1e27a7607bbcd"
server_key_iv: "6220de5fa1e508e74edf781a6a2e91cf"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.162487 24609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:45.197084 24609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:45.202169 24609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:45.202507 24609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:45.203584 24609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:45.205178 24609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:45.205286 24609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.205449 24609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:45.205525 24609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.242512 24609 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:45623
I20260807 18:09:45.242604 24739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:45623 every 8 connection(s)
I20260807 18:09:45.243908 24609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:09:45.248937 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24609
I20260807 18:09:45.249184 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:09:45.250029 8442 external_mini_cluster.cc:1428] Setting key 86d1f51b6f56220307dbc8505c2d91e7
Aug 07 18:09:45 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:45.272258 24742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:45.246718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:58409 (local address 127.8.62.190:43875)
0807 18:09:45.247230 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:45.247235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:45.248125 (+ 890us) server_negotiation.cc:407] Connection header received
0807 18:09:45.251040 (+ 2915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:45.251046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:45.251151 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:45.251287 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:45.253348 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.254164 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.256967 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.257414 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.257549 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:45.263579 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:45.263590 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:45.263597 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:45.263661 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:45.268013 (+ 4352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:45.269172 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:45.269182 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:45.269188 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:45.269368 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:45.269912 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:45.269921 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:45.269928 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:45.270383 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:45.270564 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:45.271380 (+ 816us) server_negotiation.cc:299] Negotiation successful
0807 18:09:45.271711 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":273,"threads_started":1}
I20260807 18:09:45.272953 24743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:45.247150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:58409)
0807 18:09:45.247936 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:45.247984 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:45.250657 (+ 2673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:45.251570 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:45.251579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:45.252167 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:45.253119 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:45.253152 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.254369 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.254377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:45.256766 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:45.256789 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.257714 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.257725 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:45.258099 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:45.258982 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:45.259028 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:45.263157 (+ 4129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:45.268267 (+ 5110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:45.268276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:45.268291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:45.268972 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:45.269522 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:45.269526 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:45.269529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:45.269766 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:45.270529 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:45.270536 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:45.270935 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 18:09:45.271192 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 18:09:45.271665 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":315,"threads_started":1}
I20260807 18:09:45.274745 24740 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:09:45.275110 24740 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:45.275952 24740 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:09:45.279049 24544 ts_manager.cc:194] Registered new tserver with Master: e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623)
I20260807 18:09:45.280769 24544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:58409
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:09:45.317297 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:45.670516 24746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:45.670883 24746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:45.671056 24746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:45.678670 24746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:45.678851 24746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:45.679029 24746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:09:45.689395 24746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:45.690826 24746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:45.692222 24746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:45.706523 24751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:45.706399 24752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:45.706707 24754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:45.707202 24746 server_base.cc:1034] running on GCE node
I20260807 18:09:45.707954 24746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:45.708988 24746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:45.710201 24746 hybrid_clock.cc:648] HybridClock initialized: now 1786126185710159 us; error 31 us; skew 500 ppm
I20260807 18:09:45.710547 24746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:45 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:45.713905 24746 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:09:45.715654 24746 webserver.cc:460] Webserver started at http://127.8.62.130:44199/ using document root <none> and password file <none>
I20260807 18:09:45.716445 24746 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:45.716614 24746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:45.716948 24746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:45.719957 24746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "f201064af0354bf4a13d94e9de10ec85"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "e87872e0a3dac980e7d582c63ff51aa5"
server_key_iv: "dbe549382b05c6a06a5aaa218cdafdcb"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.720841 24746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "f201064af0354bf4a13d94e9de10ec85"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "e87872e0a3dac980e7d582c63ff51aa5"
server_key_iv: "dbe549382b05c6a06a5aaa218cdafdcb"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.726186 24746 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:09:45.729837 24761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.731230 24746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:09:45.731419 24746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "f201064af0354bf4a13d94e9de10ec85"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "e87872e0a3dac980e7d582c63ff51aa5"
server_key_iv: "dbe549382b05c6a06a5aaa218cdafdcb"
server_key_version: "encryptionkey@0"
I20260807 18:09:45.731638 24746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:45.749372 24746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:45.754257 24746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:45.754592 24746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:45.755700 24746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:45.757352 24746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:45.757469 24746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.757629 24746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:45.757702 24746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:45.793753 24746 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:42689
I20260807 18:09:45.793805 24877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:42689 every 8 connection(s)
I20260807 18:09:45.795238 24746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:09:45.805426 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24746
I20260807 18:09:45.805650 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:09:45.806097 8442 external_mini_cluster.cc:1428] Setting key c25258ca89f0e3aacdffa8ec15df308f
Aug 07 18:09:45 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126185, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:45.817291 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:45.798252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:55403 (local address 127.8.62.190:43875)
0807 18:09:45.798734 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:45.798739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:45.799942 (+ 1203us) server_negotiation.cc:407] Connection header received
0807 18:09:45.801855 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:45.801861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:45.801955 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:45.802103 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:45.803661 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.804404 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.806069 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.806554 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.806704 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:45.811064 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:45.811069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:45.811073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:45.811106 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:45.813845 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:45.814740 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:45.814753 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:45.814756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:45.814885 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:45.815390 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:45.815394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:45.815396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:45.815730 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:45.815899 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:45.816663 (+ 764us) server_negotiation.cc:299] Negotiation successful
0807 18:09:45.816897 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":258,"threads_started":1}
I20260807 18:09:45.818297 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:45.798908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:55403)
0807 18:09:45.799682 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:45.799721 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:45.801566 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:45.802354 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:45.802362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:45.802915 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:45.803481 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:45.803498 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.804584 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.804590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:45.805925 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:45.805937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:45.807015 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:45.807020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:45.807295 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:45.808084 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:45.808133 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:45.810819 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:45.814016 (+ 3197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:45.814026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:45.814046 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:45.814596 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:45.815025 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:45.815031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:45.815039 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:45.815259 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:45.815866 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:45.815874 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:45.816252 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 18:09:45.816529 (+ 277us) client_negotiation.cc:240] Negotiation successful
0807 18:09:45.817125 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":294,"threads_started":1}
I20260807 18:09:45.819860 24878 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:09:45.820202 24878 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:45.821010 24878 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:09:45.822580 24544 ts_manager.cc:194] Registered new tserver with Master: f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689)
I20260807 18:09:45.823896 24544 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:55403
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:09:45.869256 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:09:46.228168 24884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:09:46.228504 24884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:09:46.228695 24884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:09:46.236426 24884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:09:46.236593 24884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:09:46.236785 24884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:09:46.247246 24884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:09:46.248636 24884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:09:46.250049 24884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:09:46.264354 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:46.264314 24893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:09:46.264391 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:09:46.265350 24884 server_base.cc:1034] running on GCE node
I20260807 18:09:46.266212 24884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:09:46.267508 24884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:09:46.268751 24884 hybrid_clock.cc:648] HybridClock initialized: now 1786126186268697 us; error 64 us; skew 500 ppm
I20260807 18:09:46.269119 24884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:09:46 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126186, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:09:46.272382 24884 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:09:46.274020 24884 webserver.cc:460] Webserver started at http://127.8.62.131:34491/ using document root <none> and password file <none>
I20260807 18:09:46.274689 24884 fs_manager.cc:359] Metadata directory not provided
I20260807 18:09:46.274791 24884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:09:46.275051 24884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:09:46.277995 24884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "a86721240cb74c33a98e603f46a70179"
format_stamp: "Formatted at 2026-08-07 18:09:46 on dist-test-slave-rnrw"
server_key: "de1ec72e87bbd97b0a82453e4734dc50"
server_key_iv: "e55e8f2c6c93f64149145a3daef150b2"
server_key_version: "encryptionkey@0"
I20260807 18:09:46.278733 24884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "a86721240cb74c33a98e603f46a70179"
format_stamp: "Formatted at 2026-08-07 18:09:46 on dist-test-slave-rnrw"
server_key: "de1ec72e87bbd97b0a82453e4734dc50"
server_key_iv: "e55e8f2c6c93f64149145a3daef150b2"
server_key_version: "encryptionkey@0"
I20260807 18:09:46.284328 24884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:09:46.284467 24740 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:09:46.288902 24901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:46.290337 24884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:09:46.290537 24884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "a86721240cb74c33a98e603f46a70179"
format_stamp: "Formatted at 2026-08-07 18:09:46 on dist-test-slave-rnrw"
server_key: "de1ec72e87bbd97b0a82453e4734dc50"
server_key_iv: "e55e8f2c6c93f64149145a3daef150b2"
server_key_version: "encryptionkey@0"
I20260807 18:09:46.290759 24884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:09:46.323837 24884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:09:46.328457 24884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:09:46.328892 24884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:09:46.329996 24884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:09:46.331462 24884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:09:46.331542 24884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:46.331687 24884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:09:46.331741 24884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:09:46.367297 24884 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:35405
I20260807 18:09:46.367334 25014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:35405 every 8 connection(s)
I20260807 18:09:46.368772 24884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:09:46.377465 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 24884
I20260807 18:09:46.377710 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:09:46.378147 8442 external_mini_cluster.cc:1428] Setting key f434ed04ad91f35120a86f146d1ef67a
Aug 07 18:09:46 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126186, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:09:46.390108 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.371465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:53345 (local address 127.8.62.190:43875)
0807 18:09:46.371987 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.371993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.373077 (+ 1084us) server_negotiation.cc:407] Connection header received
0807 18:09:46.374881 (+ 1804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.374886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.374964 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.375072 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:46.376726 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.377524 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.379456 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.379859 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.380006 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.383697 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:46.383702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:46.383705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:46.383740 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:46.386603 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.387443 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.387448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.387450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.387582 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.388062 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.388066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.388068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.388367 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:46.388557 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.389407 (+ 850us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.389681 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":242,"threads_started":1}
I20260807 18:09:46.391292 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.372111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:53345)
0807 18:09:46.372909 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.372960 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.374604 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.375292 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.375300 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.375896 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:09:46.376472 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.376490 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.377715 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.377719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.379297 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.379309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.380105 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.380112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.380363 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.381107 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:09:46.381136 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:09:46.383419 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:09:46.386772 (+ 3353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:46.386783 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:46.386803 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:46.387276 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:46.387703 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:09:46.387709 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:09:46.387713 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:09:46.387925 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:09:46.388516 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:09:46.388526 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:09:46.388987 (+ 461us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.389295 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.389936 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":297,"threads_started":1}
I20260807 18:09:46.393343 25015 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:09:46.393942 25015 heartbeater.cc:464] Registering TS with master...
I20260807 18:09:46.395166 25015 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:09:46.396911 24543 ts_manager.cc:194] Registered new tserver with Master: a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405)
I20260807 18:09:46.397845 24543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:53345
I20260807 18:09:46.407687 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:09:46.425953 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.412523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.8.62.190:43875)
0807 18:09:46.412743 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.412748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.413233 (+ 485us) server_negotiation.cc:407] Connection header received
0807 18:09:46.413362 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.413366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.413433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.413525 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:46.415176 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.416142 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.417571 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.417998 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.418099 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.420663 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:46.420668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:46.420671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:46.420705 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:46.423337 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.424091 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.424095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.424097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.424216 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.424582 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.424585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.424587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.424915 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:46.425116 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.425419 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.425592 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 18:09:46.430321 24543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:09:46.437599 24543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:09:46.461782 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.450586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.8.62.131:35405)
0807 18:09:46.453668 (+ 3082us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.453675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.453710 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:09:46.453820 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.453828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.454054 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.454853 (+ 799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:46.455440 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.456996 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.459384 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.460474 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.460680 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.460921 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.461241 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.461411 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2895,"thread_start_us":2728,"threads_started":1}
I20260807 18:09:46.461928 25031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.450348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35405 (local address 127.0.0.1:50328)
0807 18:09:46.451373 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.451428 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.451653 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.454286 (+ 2633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.454290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.454337 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:46.455202 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.455209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.457137 (+ 1928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.457144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.459216 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.459233 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.460708 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.460714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.460942 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.461011 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.461191 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.461405 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":51,"thread_start_us":1646,"threads_started":1}
I20260807 18:09:46.462816 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.451044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:45623 (local address 127.0.0.1:50608)
0807 18:09:46.452942 (+ 1898us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.452960 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.453092 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.454103 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.454111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.454159 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:46.454876 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.454900 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.456914 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.456922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.459588 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.459608 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.461903 (+ 2295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.461914 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.462111 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.462148 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.462253 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.462381 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1746,"mutex_wait_us":83,"spinlock_wait_cycles":2432,"thread_start_us":1618,"threads_started":1}
I20260807 18:09:46.463728 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.451645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50608 (local address 127.8.62.129:45623)
0807 18:09:46.452748 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.452758 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.453217 (+ 459us) server_negotiation.cc:407] Connection header received
0807 18:09:46.453302 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.453310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.453580 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.453779 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:46.455103 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.456776 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.459770 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.461336 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.461490 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.461738 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.463155 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.463373 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":708,"threads_started":1}
I20260807 18:09:46.464733 25029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.450667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:42689 (local address 127.0.0.1:55136)
0807 18:09:46.452285 (+ 1618us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.452305 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.452421 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.454165 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.454169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.454188 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:46.455195 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.455207 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.459239 (+ 4032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.459244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.461353 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.461369 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.463888 (+ 2519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.463894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.464067 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.464108 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.464199 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.464336 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1496,"mutex_wait_us":252,"spinlock_wait_cycles":12544,"thread_start_us":691,"threads_started":1}
I20260807 18:09:46.465029 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.451208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55136 (local address 127.8.62.130:42689)
0807 18:09:46.453152 (+ 1944us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.453162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.453191 (+ 29us) server_negotiation.cc:407] Connection header received
0807 18:09:46.453291 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.453299 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.453787 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.453968 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:46.456671 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.459087 (+ 2416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.461549 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.463670 (+ 2121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.463834 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.464027 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.464383 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.464573 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1794,"thread_start_us":1696,"threads_started":1}
I20260807 18:09:46.467401 24673 tablet_service.cc:1467] Processing CreateTablet for tablet d1c8dbaaa479454081df0118de237916 (DEFAULT_TABLE table=test-table [id=7f428cf59d424809a4b53bc41e37e44b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:46.466641 24949 tablet_service.cc:1467] Processing CreateTablet for tablet d1c8dbaaa479454081df0118de237916 (DEFAULT_TABLE table=test-table [id=7f428cf59d424809a4b53bc41e37e44b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:46.468072 24812 tablet_service.cc:1467] Processing CreateTablet for tablet d1c8dbaaa479454081df0118de237916 (DEFAULT_TABLE table=test-table [id=7f428cf59d424809a4b53bc41e37e44b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:09:46.469024 24949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c8dbaaa479454081df0118de237916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:46.469035 24673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c8dbaaa479454081df0118de237916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:46.469197 24812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1c8dbaaa479454081df0118de237916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:09:46.482018 25036 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Bootstrap starting.
I20260807 18:09:46.484326 25037 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Bootstrap starting.
I20260807 18:09:46.485443 25036 tablet_bootstrap.cc:654] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:46.486342 25038 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Bootstrap starting.
I20260807 18:09:46.486848 25036 log.cc:824] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:46.488667 25037 tablet_bootstrap.cc:654] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:46.489802 25038 tablet_bootstrap.cc:654] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Neither blocks nor log segments found. Creating new log.
I20260807 18:09:46.489975 25037 log.cc:824] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:46.490458 25036 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: No bootstrap required, opened a new log
I20260807 18:09:46.490789 25036 ts_tablet_manager.cc:1397] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260807 18:09:46.491091 25038 log.cc:824] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Log is configured to *not* fsync() on all Append() calls
I20260807 18:09:46.494653 25038 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: No bootstrap required, opened a new log
I20260807 18:09:46.495005 25038 ts_tablet_manager.cc:1397] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 18:09:46.495784 25037 tablet_bootstrap.cc:492] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: No bootstrap required, opened a new log
I20260807 18:09:46.496052 25037 ts_tablet_manager.cc:1397] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260807 18:09:46.504222 25038 raft_consensus.cc:348] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.504324 25036 raft_consensus.cc:348] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.504691 25038 raft_consensus.cc:374] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:46.504787 25038 raft_consensus.cc:729] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86721240cb74c33a98e603f46a70179, State: Initialized, Role: FOLLOWER
I20260807 18:09:46.504815 25036 raft_consensus.cc:374] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:46.504916 25036 raft_consensus.cc:729] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f201064af0354bf4a13d94e9de10ec85, State: Initialized, Role: FOLLOWER
I20260807 18:09:46.505506 25038 consensus_queue.cc:260] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.505656 25036 consensus_queue.cc:260] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.506709 25015 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:09:46.507534 25038 ts_tablet_manager.cc:1428] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260807 18:09:46.507746 25037 raft_consensus.cc:348] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.508277 25037 raft_consensus.cc:374] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:09:46.508388 25037 raft_consensus.cc:729] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bfe365d3904531b2ce0c5d0577f09e, State: Initialized, Role: FOLLOWER
I20260807 18:09:46.509019 24878 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:09:46.509438 25036 ts_tablet_manager.cc:1428] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Time spent starting tablet: real 0.019s user 0.014s sys 0.004s
I20260807 18:09:46.509193 25037 consensus_queue.cc:260] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.511164 25037 ts_tablet_manager.cc:1428] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
W20260807 18:09:46.547501 24879 tablet.cc:2369] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:46.592240 25043 raft_consensus.cc:482] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:09:46.592756 25043 raft_consensus.cc:504] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.594636 25043 leader_election.cc:288] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405), e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623)
I20260807 18:09:46.602443 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.595416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:52533 (local address 127.8.62.131:35405)
0807 18:09:46.595674 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.595679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.596175 (+ 496us) server_negotiation.cc:407] Connection header received
0807 18:09:46.596378 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.596386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.596473 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.596594 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:46.597438 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.598271 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.600331 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.601287 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.601421 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.601479 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.602016 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.602180 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 18:09:46.602448 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.595129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:36657 (local address 127.8.62.129:45623)
0807 18:09:46.595369 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.595379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.596496 (+ 1117us) server_negotiation.cc:407] Connection header received
0807 18:09:46.596767 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.596772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.596871 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.596996 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:09:46.597735 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.598642 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.600507 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.601465 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.601643 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.601749 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.602008 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.602161 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:09:46.602624 25045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.594884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35405 (local address 127.8.62.130:52533)
0807 18:09:46.596014 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.596045 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.596220 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.596723 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.596728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.596757 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:46.597259 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.597275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.598416 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.598422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.600139 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.600152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.601540 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.601550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.601805 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.601865 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.601978 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.602200 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":117,"thread_start_us":682,"threads_started":1}
I20260807 18:09:46.603076 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.595109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:45623 (local address 127.8.62.130:36657)
0807 18:09:46.596288 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0807 18:09:46.596309 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:09:46.596460 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:09:46.597020 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:09:46.597023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:09:46.597047 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:09:46.597533 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.597546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.598804 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.598808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.600331 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:09:46.600347 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.601684 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.601688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:09:46.601847 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.601870 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:09:46.601980 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 18:09:46.602158 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"mutex_wait_us":9,"thread_start_us":732,"threads_started":1}
I20260807 18:09:46.603894 24970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c8dbaaa479454081df0118de237916" candidate_uuid: "f201064af0354bf4a13d94e9de10ec85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86721240cb74c33a98e603f46a70179" is_pre_election: true
I20260807 18:09:46.604029 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c8dbaaa479454081df0118de237916" candidate_uuid: "f201064af0354bf4a13d94e9de10ec85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" is_pre_election: true
I20260807 18:09:46.604458 24970 raft_consensus.cc:2454] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f201064af0354bf4a13d94e9de10ec85 in term 0.
I20260807 18:09:46.604696 24693 raft_consensus.cc:2454] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f201064af0354bf4a13d94e9de10ec85 in term 0.
I20260807 18:09:46.605329 24764 leader_election.cc:302] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a86721240cb74c33a98e603f46a70179, f201064af0354bf4a13d94e9de10ec85; no voters:
I20260807 18:09:46.605832 25043 raft_consensus.cc:2781] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:09:46.606002 25043 raft_consensus.cc:482] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:09:46.606117 25043 raft_consensus.cc:3037] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:46.608656 25043 raft_consensus.cc:504] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
I20260807 18:09:46.609407 25043 leader_election.cc:288] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [CANDIDATE]: Term 1 election: Requested vote from peers a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405), e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623)
I20260807 18:09:46.609890 24970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c8dbaaa479454081df0118de237916" candidate_uuid: "f201064af0354bf4a13d94e9de10ec85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86721240cb74c33a98e603f46a70179"
I20260807 18:09:46.610014 24693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1c8dbaaa479454081df0118de237916" candidate_uuid: "f201064af0354bf4a13d94e9de10ec85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
I20260807 18:09:46.610239 24693 raft_consensus.cc:3037] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:46.610188 24970 raft_consensus.cc:3037] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:09:46.612596 24693 raft_consensus.cc:2454] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f201064af0354bf4a13d94e9de10ec85 in term 1.
I20260807 18:09:46.612637 24970 raft_consensus.cc:2454] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f201064af0354bf4a13d94e9de10ec85 in term 1.
I20260807 18:09:46.613111 24762 leader_election.cc:302] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bfe365d3904531b2ce0c5d0577f09e, f201064af0354bf4a13d94e9de10ec85; no voters:
I20260807 18:09:46.613446 25043 raft_consensus.cc:2781] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:09:46.613982 25043 raft_consensus.cc:686] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 1 LEADER]: Becoming Leader. State: Replica: f201064af0354bf4a13d94e9de10ec85, State: Running, Role: LEADER
I20260807 18:09:46.614639 25043 consensus_queue.cc:237] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } }
W20260807 18:09:46.620486 25016 tablet.cc:2369] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:09:46.620167 24544 catalog_manager.cc:5495] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 reported cstate change: term changed from 0 to 1, leader changed from <none> to f201064af0354bf4a13d94e9de10ec85 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "f201064af0354bf4a13d94e9de10ec85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } health_report { overall_health: UNKNOWN } } }
I20260807 18:09:46.656196 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.650719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.8.62.130:42689)
0807 18:09:46.650953 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.650961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.650988 (+ 27us) server_negotiation.cc:407] Connection header received
0807 18:09:46.651085 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.651089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.651168 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.651333 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:09:46.651742 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.652480 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.653937 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.654380 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.654519 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.655027 (+ 508us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:09:46.655578 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:09:46.655751 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.655805 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.655889 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:09:46.664880 24693 raft_consensus.cc:1261] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Refusing update from remote peer f201064af0354bf4a13d94e9de10ec85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:46.664889 24970 raft_consensus.cc:1261] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Refusing update from remote peer f201064af0354bf4a13d94e9de10ec85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:09:46.665937 25047 consensus_queue.cc:1035] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:46.666330 25043 consensus_queue.cc:1035] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:09:46.692937 24543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:46.693286 24543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:09:46.697849 24543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Sending DeleteTablet for 3 replicas of tablet d1c8dbaaa479454081df0118de237916
I20260807 18:09:46.699318 24812 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c8dbaaa479454081df0118de237916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:55136
I20260807 18:09:46.699314 24673 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c8dbaaa479454081df0118de237916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:50608
I20260807 18:09:46.699559 24949 tablet_service.cc:1514] Processing DeleteTablet for tablet d1c8dbaaa479454081df0118de237916 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:09:46 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:50328
I20260807 18:09:46.701093 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24609
I20260807 18:09:46.716439 25061 tablet_replica.cc:330] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: stopping tablet replica
I20260807 18:09:46.715525 25062 tablet_replica.cc:330] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: stopping tablet replica
I20260807 18:09:46.717514 25062 raft_consensus.cc:2229] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:09:46.718492 25061 raft_consensus.cc:2229] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:46.719467 25062 raft_consensus.cc:2258] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:46.720232 25061 raft_consensus.cc:2258] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:46.721314 25063 tablet_replica.cc:330] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: stopping tablet replica
I20260807 18:09:46.721774 25063 raft_consensus.cc:2229] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:09:46.722357 25063 raft_consensus.cc:2258] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:09:46.724076 25062 ts_tablet_manager.cc:1905] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:46.724702 25063 ts_tablet_manager.cc:1905] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:46.725863 25061 ts_tablet_manager.cc:1905] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:09:46.732630 25063 ts_tablet_manager.cc:1918] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:46.732841 25063 log.cc:1196] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/d1c8dbaaa479454081df0118de237916
Aug 07 18:09:46 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:09:46.733410 25063 ts_tablet_manager.cc:1939] T d1c8dbaaa479454081df0118de237916 P a86721240cb74c33a98e603f46a70179: Deleting consensus metadata
I20260807 18:09:46.735587 24530 catalog_manager.cc:4841] TS a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405): tablet d1c8dbaaa479454081df0118de237916 (table test-table [id=7f428cf59d424809a4b53bc41e37e44b]) successfully deleted
I20260807 18:09:46.739990 25062 ts_tablet_manager.cc:1918] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:46.740232 25062 log.cc:1196] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/d1c8dbaaa479454081df0118de237916
I20260807 18:09:46.740867 25061 ts_tablet_manager.cc:1918] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:09:46.740949 25062 ts_tablet_manager.cc:1939] T d1c8dbaaa479454081df0118de237916 P f201064af0354bf4a13d94e9de10ec85: Deleting consensus metadata
I20260807 18:09:46.741194 25061 log.cc:1196] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/d1c8dbaaa479454081df0118de237916
I20260807 18:09:46.741889 25061 ts_tablet_manager.cc:1939] T d1c8dbaaa479454081df0118de237916 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting consensus metadata
I20260807 18:09:46.743155 24529 catalog_manager.cc:4841] TS f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689): tablet d1c8dbaaa479454081df0118de237916 (table test-table [id=7f428cf59d424809a4b53bc41e37e44b]) successfully deleted
I20260807 18:09:46.744015 24528 catalog_manager.cc:4841] TS e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623): tablet d1c8dbaaa479454081df0118de237916 (table test-table [id=7f428cf59d424809a4b53bc41e37e44b]) successfully deleted
I20260807 18:09:46.744130 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.704216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50622 (local address 127.8.62.129:45623)
0807 18:09:46.704483 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.704493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.719425 (+ 14932us) server_negotiation.cc:407] Connection header received
0807 18:09:46.719983 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.719988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.720097 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.720179 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:46.723413 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.724596 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.726973 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.727709 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.727829 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.734899 (+ 7070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:46.734936 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:46.734959 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:46.735020 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:46.739633 (+ 4613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.740937 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.740950 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.740954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.741155 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.741771 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.741776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.741780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.742334 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:46.742675 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.743410 (+ 735us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.743691 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:09:46.745699 24735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:46.848753 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24609
I20260807 18:09:46.870195 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24746
Aug 07 18:09:46 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:09:46.891328 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:46.871212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.8.62.130:42689)
0807 18:09:46.871513 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:46.871521 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:46.871554 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:09:46.871787 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:46.871792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:46.871868 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:46.871980 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:46.874019 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.875283 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.877689 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:46.878242 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:46.878368 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:46.884186 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:46.884205 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:46.884222 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:46.884265 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:46.887942 (+ 3677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.889034 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.889042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.889044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.889225 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:46.889707 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:46.889710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:46.889712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:46.890111 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:46.890363 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:46.890792 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 18:09:46.890981 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:09:46.892102 24873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:47.013931 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24746
I20260807 18:09:47.034611 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24884
Aug 07 18:09:47 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:09:47.052001 25030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:09:47.035567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50338 (local address 127.8.62.131:35405)
0807 18:09:47.035848 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 18:09:47.035853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:09:47.035871 (+ 18us) server_negotiation.cc:407] Connection header received
0807 18:09:47.035929 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:09:47.035933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:09:47.036012 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:09:47.036103 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:09:47.037731 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:47.038517 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:47.040021 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:09:47.040450 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:09:47.040729 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:09:47.045642 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:09:47.045661 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:09:47.045679 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:09:47.045725 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:09:47.049046 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:47.049889 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:47.049897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:47.049899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:47.050098 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:09:47.050454 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:09:47.050457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:09:47.050459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:09:47.050863 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:09:47.051120 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0807 18:09:47.051441 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 18:09:47.051622 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260807 18:09:47.052861 25010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:47.145781 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24884
I20260807 18:09:47.164439 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 24507
I20260807 18:09:47.165838 24577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:09:47.281464 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 24507
I20260807 18:10:07.306340 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--webserver_interface=127.8.62.190
--webserver_port=41415
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:43875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:07.673053 25069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:07.673341 25069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:07.673430 25069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:07.680658 25069 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:10:07.680763 25069 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:10:07.680802 25069 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:10:07.680857 25069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:10:07.680912 25069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:07.680974 25069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:07.681022 25069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:10:07.681071 25069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:10:07.691035 25069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:43875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:07.692276 25069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:07.693576 25069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:07.719542 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:07.719497 25079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:07.719594 25077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:07.724749 25069 server_base.cc:1034] running on GCE node
I20260807 18:10:07.725808 25069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:07.727653 25069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:07.728935 25069 hybrid_clock.cc:648] HybridClock initialized: now 1786126207728864 us; error 47 us; skew 500 ppm
I20260807 18:10:07.729364 25069 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:07 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126207, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:07.733520 25069 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:10:07.735536 25069 webserver.cc:460] Webserver started at http://127.8.62.190:41415/ using document root <none> and password file <none>
I20260807 18:10:07.736442 25069 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:07.736647 25069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:07.743587 25069 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260807 18:10:07.747169 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:07.749081 25069 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:10:07.749327 25069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "55b5965c6aae4f5f8bb6ef8369b22d75"
format_stamp: "Formatted at 2026-08-07 18:09:44 on dist-test-slave-rnrw"
server_key: "176f5f215a280f85ae29c469a2b8f7fd"
server_key_iv: "3339d3e80c31ec2e09fad16d1993eae9"
server_key_version: "encryptionkey@0"
I20260807 18:10:07.750527 25069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:07.768314 25069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:07.773159 25069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:07.773494 25069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:07.794873 25069 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:43875
I20260807 18:10:07.794893 25138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:43875 every 8 connection(s)
I20260807 18:10:07.796720 25069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:10:07.804374 25139 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:10:07.805931 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25069
I20260807 18:10:07.806852 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:45623
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=37125
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:10:07.808197 25139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap starting.
I20260807 18:10:07.830487 25139 log.cc:824] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:07.843312 25139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:10:07.843879 25139 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap complete.
I20260807 18:10:07.853848 25139 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:07.854885 25139 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Initialized, Role: FOLLOWER
I20260807 18:10:07.855634 25139 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:07.855955 25139 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:10:07.856112 25139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:10:07.856316 25139 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 1 FOLLOWER]: Advancing to term 2
I20260807 18:10:07.860818 25139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:07.861442 25139 leader_election.cc:302] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b5965c6aae4f5f8bb6ef8369b22d75; no voters:
I20260807 18:10:07.862178 25139 leader_election.cc:288] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 18:10:07.862341 25143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 18:10:07.863708 25143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 LEADER]: Becoming Leader. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Running, Role: LEADER
I20260807 18:10:07.864419 25143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:07.864868 25139 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:10:07.867215 25145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b5965c6aae4f5f8bb6ef8369b22d75. Latest consensus state: current_term: 2 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:10:07.867630 25145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:07.867522 25144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:10:07.867957 25144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:07.878064 25150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:10:07.887086 25150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:10:07.888826 25150 catalog_manager.cc:1259] Loaded cluster ID: 8dcdf638728c4bc29386f78203d99707
I20260807 18:10:07.889035 25150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:10:07.894354 25150 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:10:07.896183 25150 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Loaded TSK: 0
I20260807 18:10:07.943009 25150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Generated new TSK 1
I20260807 18:10:07.943470 25150 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 18:10:08.199451 25141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:08.199775 25141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:08.199934 25141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:08.208135 25141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:08.208266 25141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:08.208441 25141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:10:08.219142 25141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=37125
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:08.220607 25141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:08.222066 25141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:08.239275 25168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:08.239275 25169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:08.240464 25171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:08.240474 25141 server_base.cc:1034] running on GCE node
I20260807 18:10:08.241365 25141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:08.242491 25141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:08.243775 25141 hybrid_clock.cc:648] HybridClock initialized: now 1786126208243703 us; error 68 us; skew 500 ppm
I20260807 18:10:08.244171 25141 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:08 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:08.248029 25141 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:10:08.249879 25141 webserver.cc:460] Webserver started at http://127.8.62.129:37125/ using document root <none> and password file <none>
I20260807 18:10:08.250649 25141 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:08.250823 25141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:08.257057 25141 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 18:10:08.260210 25178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.261937 25141 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:10:08.262142 25141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "acfbdf31457c08292df1e27a7607bbcd"
server_key_iv: "6220de5fa1e508e74edf781a6a2e91cf"
server_key_version: "encryptionkey@0"
I20260807 18:10:08.263183 25141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:08.278825 25141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:08.289162 25141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:08.289520 25141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:08.290655 25141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:08.292217 25141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:08.292320 25141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.292454 25141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:08.292584 25141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.329290 25141 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:45623
I20260807 18:10:08.329384 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:45623 every 8 connection(s)
I20260807 18:10:08.330837 25141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:10:08.336979 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25141
I20260807 18:10:08.337685 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:42689
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=44199
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:10:08 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:08.361511 25298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:08.334439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:54057)
0807 18:10:08.335415 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:08.335464 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:08.337446 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:08.339330 (+ 1884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:08.339339 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:08.339935 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:08.340537 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:08.340636 (+ 99us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.343248 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.343253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:08.344640 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:08.344662 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.346359 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.346366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:08.346677 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:08.347549 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:08.347588 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:08.350593 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:08.356209 (+ 5616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:08.356218 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:08.356233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:08.356878 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:08.357545 (+ 667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:08.357549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:08.357551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:08.357776 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:08.358952 (+ 1176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:08.358959 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:08.359372 (+ 413us) client_negotiation.cc:769] Sending connection context
0807 18:10:08.359667 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 18:10:08.360221 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":427,"threads_started":1}
I20260807 18:10:08.362496 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:08.334116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:54057 (local address 127.8.62.190:43875)
0807 18:10:08.334726 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:08.334740 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:08.335650 (+ 910us) server_negotiation.cc:407] Connection header received
0807 18:10:08.338144 (+ 2494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:08.338166 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:08.338686 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:08.339177 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:08.340894 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.343030 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.344813 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.346055 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.346295 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:08.351035 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:08.351050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:08.351076 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:08.351144 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:08.355986 (+ 4842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:08.357085 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:08.357099 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:08.357111 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:08.357386 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:08.357937 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:08.357945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:08.357951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:08.358768 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:08.359011 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:08.360021 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0807 18:10:08.360599 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":259,"threads_started":1}
I20260807 18:10:08.366189 25295 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:10:08.366665 25295 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:08.367609 25295 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:10:08.370512 25104 ts_manager.cc:194] Registered new tserver with Master: e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623)
I20260807 18:10:08.372910 25104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:54057
W20260807 18:10:08.697211 25299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:08.697527 25299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:08.697625 25299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:08.705286 25299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:08.705415 25299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:08.705534 25299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:10:08.715947 25299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=44199
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:08.717279 25299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:08.718514 25299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:08.732435 25304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:08.732468 25308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:08.732533 25305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:08.733574 25299 server_base.cc:1034] running on GCE node
I20260807 18:10:08.734231 25299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:08.735184 25299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:08.736413 25299 hybrid_clock.cc:648] HybridClock initialized: now 1786126208736339 us; error 58 us; skew 500 ppm
I20260807 18:10:08.736812 25299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:08 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:08.740175 25299 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:10:08.741991 25299 webserver.cc:460] Webserver started at http://127.8.62.130:44199/ using document root <none> and password file <none>
I20260807 18:10:08.742760 25299 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:08.742918 25299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:08.749372 25299 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 18:10:08.752717 25316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.754725 25299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 18:10:08.754947 25299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "f201064af0354bf4a13d94e9de10ec85"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "e87872e0a3dac980e7d582c63ff51aa5"
server_key_iv: "dbe549382b05c6a06a5aaa218cdafdcb"
server_key_version: "encryptionkey@0"
I20260807 18:10:08.756094 25299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:08.791544 25299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:08.796305 25299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:08.796711 25299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:08.797827 25299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:08.799309 25299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:08.799400 25299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.799568 25299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:08.799654 25299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:08.836079 25299 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:42689
I20260807 18:10:08.836170 25435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:42689 every 8 connection(s)
I20260807 18:10:08.837682 25299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:10:08.842599 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25299
I20260807 18:10:08.843235 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:35405
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=34491
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 18:10:08 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126208, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:08.859073 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:08.840385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45731 (local address 127.8.62.190:43875)
0807 18:10:08.840635 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:08.840642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:08.841798 (+ 1156us) server_negotiation.cc:407] Connection header received
0807 18:10:08.843843 (+ 2045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:08.843850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:08.843960 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:08.844084 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:08.845768 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.846500 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.848129 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.848621 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.848757 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:08.852812 (+ 4055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:08.852817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:08.852820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:08.852857 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:08.855677 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:08.856516 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:08.856523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:08.856531 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:08.856742 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:08.857232 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:08.857237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:08.857240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:08.857552 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:08.857717 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:08.858461 (+ 744us) server_negotiation.cc:299] Negotiation successful
0807 18:10:08.858713 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:10:08.860298 25438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:08.840889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:45731)
0807 18:10:08.841627 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:08.841672 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:08.843544 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:08.844346 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:08.844355 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:08.844979 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:08.845570 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:08.845599 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.846689 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.846693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:08.847969 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:08.847981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:08.848854 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:08.848862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:08.849109 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:08.849820 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:08.849843 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:08.852482 (+ 2639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:08.855847 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:08.855858 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:08.855878 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:08.856370 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:08.856863 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:08.856868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:08.856872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:08.857101 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:08.857688 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:08.857697 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:08.858085 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 18:10:08.858370 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 18:10:08.858966 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":298,"threads_started":1}
I20260807 18:10:08.861980 25436 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:10:08.862344 25436 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:08.863094 25436 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:10:08.864650 25104 ts_manager.cc:194] Registered new tserver with Master: f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689)
I20260807 18:10:08.865525 25104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:45731
W20260807 18:10:09.199307 25439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:09.199683 25439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:09.199868 25439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:09.207528 25439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:09.207718 25439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:09.207897 25439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:10:09.218168 25439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:35405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=34491
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:09.219632 25439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:09.221074 25439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:09.235036 25444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:09.235036 25447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:09.235652 25445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:09.235975 25439 server_base.cc:1034] running on GCE node
I20260807 18:10:09.236690 25439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:09.237679 25439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:09.238962 25439 hybrid_clock.cc:648] HybridClock initialized: now 1786126209238878 us; error 63 us; skew 500 ppm
I20260807 18:10:09.239329 25439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:09 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:09.242830 25439 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:10:09.244650 25439 webserver.cc:460] Webserver started at http://127.8.62.131:34491/ using document root <none> and password file <none>
I20260807 18:10:09.245390 25439 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:09.245527 25439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:09.251387 25439 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 18:10:09.254583 25455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:09.256273 25439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:10:09.256469 25439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "a86721240cb74c33a98e603f46a70179"
format_stamp: "Formatted at 2026-08-07 18:09:46 on dist-test-slave-rnrw"
server_key: "de1ec72e87bbd97b0a82453e4734dc50"
server_key_iv: "e55e8f2c6c93f64149145a3daef150b2"
server_key_version: "encryptionkey@0"
I20260807 18:10:09.257665 25439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:09.279131 25439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:09.284148 25439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:09.284471 25439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:09.285607 25439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:09.287088 25439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:09.287180 25439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:09.287336 25439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:09.287446 25439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:09.322909 25439 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:35405
I20260807 18:10:09.323015 25568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:35405 every 8 connection(s)
I20260807 18:10:09.324479 25439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:10:09.331979 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25439
Aug 07 18:10:09 dist-test-slave-rnrw krb5kdc[24494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126209, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:09.352375 25573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.333111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.8.62.190:43875)
0807 18:10:09.333592 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.333598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.334193 (+ 595us) server_negotiation.cc:407] Connection header received
0807 18:10:09.334372 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.334377 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.334453 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.334576 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.336579 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.337409 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.339211 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.339884 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.339975 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.343898 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.343903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.343907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.343940 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.348958 (+ 5018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.349964 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.349971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.349975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.350174 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.350682 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.350687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.350689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.351153 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.351449 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.351776 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.351957 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":368,"spinlock_wait_cycles":6400,"thread_start_us":254,"threads_started":1}
I20260807 18:10:09.352982 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.327221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:36951 (local address 127.8.62.190:43875)
0807 18:10:09.327410 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.327414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.328861 (+ 1447us) server_negotiation.cc:407] Connection header received
0807 18:10:09.332326 (+ 3465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.332333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.332433 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.332609 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.335766 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.336675 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.338895 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.339457 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.339599 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.344150 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.344157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.344168 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.344219 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.348910 (+ 4691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.349964 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.349971 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.349975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.350174 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.350817 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.350825 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.350830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.351154 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.351335 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.352306 (+ 971us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.352597 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":3712}
I20260807 18:10:09.354336 25571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.327723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:36951)
0807 18:10:09.328666 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.328727 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.331314 (+ 2587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.333804 (+ 2490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.333819 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.334651 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:09.335554 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.335583 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.336860 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.336865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.338740 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.338756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.339711 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.339716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.339916 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.340758 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:09.340804 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:09.343927 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:09.349125 (+ 5198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:09.349137 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:09.349158 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:09.349833 (+ 675us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:09.350308 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:09.350313 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:09.350317 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:09.350659 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:09.351299 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:09.351306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:09.351849 (+ 543us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.352187 (+ 338us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.352798 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":338,"threads_started":1}
I20260807 18:10:09.355897 25569 heartbeater.cc:347] Connected to a master server at 127.8.62.190:43875
I20260807 18:10:09.356238 25569 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:09.357090 25569 heartbeater.cc:511] Master 127.8.62.190:43875 requested a full tablet report, sending...
I20260807 18:10:09.358608 25103 ts_manager.cc:194] Registered new tserver with Master: a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405)
I20260807 18:10:09.359493 25103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:36951
I20260807 18:10:09.368108 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:10:09.375927 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.369326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.8.62.190:43875)
0807 18:10:09.369536 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.369540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.370159 (+ 619us) server_negotiation.cc:407] Connection header received
0807 18:10:09.370375 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.370380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.370521 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.370685 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:10:09.371222 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.372067 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.373765 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.374227 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.374331 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.374988 (+ 657us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:10:09.375296 (+ 308us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 18:10:09.375711 (+ 415us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48186: token expired
Metrics: {"server-negotiator.queue_time_us":91}
W20260807 18:10:09.376197 25297 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48186: token expired
W20260807 18:10:09.376760 25022 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.62.190:43875)
I20260807 18:10:09.376804 25295 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:10:09.392351 25297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.377433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.8.62.190:43875)
0807 18:10:09.377659 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.377663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.377679 (+ 16us) server_negotiation.cc:407] Connection header received
0807 18:10:09.377742 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.377746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.377804 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.377886 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.379761 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.380682 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.382173 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.382611 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.382724 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.385854 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.385860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.385863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.385906 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.389027 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.389982 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.389988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.389990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.390151 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.390580 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.390583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.390585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.390998 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.391215 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.391748 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.391951 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:10:09.396618 25103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:10:09.404982 25103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:10:09.435891 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.420372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:45623 (local address 127.0.0.1:49858)
0807 18:10:09.421842 (+ 1470us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.421859 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.421988 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.423509 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.423521 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.423579 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:09.424178 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.424196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.427292 (+ 3096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.427303 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.430764 (+ 3461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.430797 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.434512 (+ 3715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.434520 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.434795 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.434977 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.435123 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.435374 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1320,"mutex_wait_us":58,"thread_start_us":1776,"threads_started":1}
I20260807 18:10:09.436439 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.421578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49858 (local address 127.8.62.129:45623)
0807 18:10:09.422264 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.422273 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.422304 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:10:09.422428 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.422440 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.422817 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.423096 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:09.424482 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.426848 (+ 2366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.431123 (+ 4275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.433779 (+ 2656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.434000 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.434303 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.435868 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.436069 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":347,"threads_started":1}
I20260807 18:10:09.441668 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.421066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.8.62.130:42689)
0807 18:10:09.433953 (+ 12887us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.433967 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.434009 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:10:09.434125 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.434134 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.434446 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.435456 (+ 1010us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:09.435734 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.437250 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.439371 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.440409 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.440615 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.440852 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.441187 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.441359 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12639,"thread_start_us":12519,"threads_started":1}
I20260807 18:10:09.441730 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.420260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:42689 (local address 127.0.0.1:33050)
0807 18:10:09.421144 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.421219 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.421449 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.434699 (+ 13250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.434704 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.434740 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:09.435232 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.435247 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.437403 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.437408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.439149 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.439164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.440771 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.440778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.440977 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.441008 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.441140 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.441296 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":3,"thread_start_us":415,"threads_started":1}
I20260807 18:10:09.441344 25228 tablet_service.cc:1467] Processing CreateTablet for tablet a274a180e0be41b79354c85c0f6189a0 (DEFAULT_TABLE table=test-table [id=559ada67fb804460b16d5dd77320c664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:09.443358 25228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a274a180e0be41b79354c85c0f6189a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:09.444283 25583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.421144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35405 (local address 127.0.0.1:42238)
0807 18:10:09.433101 (+ 11957us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.433130 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.433300 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.435433 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.435437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.435473 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:09.435890 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.435902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.439077 (+ 3175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.439082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.441259 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.441275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.443421 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.443428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.443603 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.443644 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.443731 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.443878 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11762,"thread_start_us":13116,"threads_started":1}
I20260807 18:10:09.444351 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.421919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.8.62.131:35405)
0807 18:10:09.434370 (+ 12451us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.434379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.434415 (+ 36us) server_negotiation.cc:407] Connection header received
0807 18:10:09.434602 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.434610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.434887 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.435107 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:09.436123 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.438872 (+ 2749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.441485 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.443171 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.443323 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.443750 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.443832 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.443997 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12259,"thread_start_us":12095,"threads_started":1}
I20260807 18:10:09.446352 25368 tablet_service.cc:1467] Processing CreateTablet for tablet a274a180e0be41b79354c85c0f6189a0 (DEFAULT_TABLE table=test-table [id=559ada67fb804460b16d5dd77320c664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:09.447808 25368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a274a180e0be41b79354c85c0f6189a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:09.449177 25503 tablet_service.cc:1467] Processing CreateTablet for tablet a274a180e0be41b79354c85c0f6189a0 (DEFAULT_TABLE table=test-table [id=559ada67fb804460b16d5dd77320c664]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:09.451632 25503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a274a180e0be41b79354c85c0f6189a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:09.461491 25586 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Bootstrap starting.
I20260807 18:10:09.463618 25587 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Bootstrap starting.
I20260807 18:10:09.466703 25586 tablet_bootstrap.cc:654] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:09.467638 25587 tablet_bootstrap.cc:654] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:09.468437 25586 log.cc:824] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:09.469506 25587 log.cc:824] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:09.470474 25589 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Bootstrap starting.
I20260807 18:10:09.472688 25586 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: No bootstrap required, opened a new log
I20260807 18:10:09.473017 25586 ts_tablet_manager.cc:1397] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.002s
I20260807 18:10:09.474745 25589 tablet_bootstrap.cc:654] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:09.475023 25587 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: No bootstrap required, opened a new log
I20260807 18:10:09.475350 25587 ts_tablet_manager.cc:1397] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:10:09.476002 25589 log.cc:824] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:09.478958 25589 tablet_bootstrap.cc:492] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: No bootstrap required, opened a new log
I20260807 18:10:09.479298 25589 ts_tablet_manager.cc:1397] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260807 18:10:09.484627 25586 raft_consensus.cc:348] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.485260 25586 raft_consensus.cc:374] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:09.485402 25586 raft_consensus.cc:729] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f201064af0354bf4a13d94e9de10ec85, State: Initialized, Role: FOLLOWER
I20260807 18:10:09.486176 25586 consensus_queue.cc:260] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.486789 25587 raft_consensus.cc:348] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.487388 25587 raft_consensus.cc:374] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:09.487550 25587 raft_consensus.cc:729] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7bfe365d3904531b2ce0c5d0577f09e, State: Initialized, Role: FOLLOWER
I20260807 18:10:09.487751 25436 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:10:09.487884 25589 raft_consensus.cc:348] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.488317 25589 raft_consensus.cc:374] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:09.488416 25589 raft_consensus.cc:729] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86721240cb74c33a98e603f46a70179, State: Initialized, Role: FOLLOWER
I20260807 18:10:09.488488 25586 ts_tablet_manager.cc:1428] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260807 18:10:09.488337 25587 consensus_queue.cc:260] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.489172 25589 consensus_queue.cc:260] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.490749 25587 ts_tablet_manager.cc:1428] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 18:10:09.491864 25569 heartbeater.cc:503] Master 127.8.62.190:43875 was elected leader, sending a full tablet report...
I20260807 18:10:09.492566 25589 ts_tablet_manager.cc:1428] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
I20260807 18:10:09.541620 25593 raft_consensus.cc:482] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:10:09.541973 25593 raft_consensus.cc:504] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.543814 25593 leader_election.cc:288] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689), a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405)
I20260807 18:10:09.550765 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.544529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:55259 (local address 127.8.62.130:42689)
0807 18:10:09.544905 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.544913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.545337 (+ 424us) server_negotiation.cc:407] Connection header received
0807 18:10:09.545602 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.545607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.545669 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.545774 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:09.546371 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.547104 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.549012 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.549788 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.549975 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.550057 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.550360 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.550487 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:10:09.550839 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.544324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:35513 (local address 127.8.62.131:35405)
0807 18:10:09.544641 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.544651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.545598 (+ 947us) server_negotiation.cc:407] Connection header received
0807 18:10:09.545820 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.545824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.545882 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.545983 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:09.546594 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.547296 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.549116 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.549878 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.550080 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.550159 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.550451 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.550582 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:10:09.550864 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.544170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:42689 (local address 127.8.62.129:55259)
0807 18:10:09.545209 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.545230 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.545437 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.545801 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.545806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.545834 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:09.546218 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.546229 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.547246 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.547251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.548844 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.548856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.550018 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.550025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.550189 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.550215 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.550332 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.550477 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":946,"threads_started":1}
I20260807 18:10:09.550984 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.544170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35405 (local address 127.8.62.129:35513)
0807 18:10:09.545474 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:09.545488 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:09.545660 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:09.546009 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:09.546014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:09.546041 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:09.546434 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.546442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.547449 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.547456 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.548944 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:09.548957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.550128 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.550134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:09.550291 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.550314 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:10:09.550422 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 18:10:09.550568 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1185,"mutex_wait_us":64,"thread_start_us":580,"threads_started":1}
I20260807 18:10:09.552062 25524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a274a180e0be41b79354c85c0f6189a0" candidate_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86721240cb74c33a98e603f46a70179" is_pre_election: true
I20260807 18:10:09.552337 25388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a274a180e0be41b79354c85c0f6189a0" candidate_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f201064af0354bf4a13d94e9de10ec85" is_pre_election: true
I20260807 18:10:09.552774 25524 raft_consensus.cc:2454] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7bfe365d3904531b2ce0c5d0577f09e in term 0.
I20260807 18:10:09.552892 25388 raft_consensus.cc:2454] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7bfe365d3904531b2ce0c5d0577f09e in term 0.
I20260807 18:10:09.553565 25181 leader_election.cc:302] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bfe365d3904531b2ce0c5d0577f09e, f201064af0354bf4a13d94e9de10ec85; no voters:
I20260807 18:10:09.553925 25593 raft_consensus.cc:2781] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:10:09.554123 25593 raft_consensus.cc:482] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:10:09.554241 25593 raft_consensus.cc:3037] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:09.556774 25593 raft_consensus.cc:504] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.557467 25593 leader_election.cc:288] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [CANDIDATE]: Term 1 election: Requested vote from peers f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689), a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405)
I20260807 18:10:09.558039 25388 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a274a180e0be41b79354c85c0f6189a0" candidate_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f201064af0354bf4a13d94e9de10ec85"
I20260807 18:10:09.558173 25524 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a274a180e0be41b79354c85c0f6189a0" candidate_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a86721240cb74c33a98e603f46a70179"
I20260807 18:10:09.558274 25388 raft_consensus.cc:3037] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:09.558468 25524 raft_consensus.cc:3037] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:09.561122 25388 raft_consensus.cc:2454] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7bfe365d3904531b2ce0c5d0577f09e in term 1.
I20260807 18:10:09.561460 25524 raft_consensus.cc:2454] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7bfe365d3904531b2ce0c5d0577f09e in term 1.
I20260807 18:10:09.561579 25181 leader_election.cc:302] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7bfe365d3904531b2ce0c5d0577f09e, f201064af0354bf4a13d94e9de10ec85; no voters:
I20260807 18:10:09.561975 25593 raft_consensus.cc:2781] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:09.562433 25593 raft_consensus.cc:686] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 LEADER]: Becoming Leader. State: Replica: e7bfe365d3904531b2ce0c5d0577f09e, State: Running, Role: LEADER
I20260807 18:10:09.563149 25593 consensus_queue.cc:237] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } }
I20260807 18:10:09.568324 25102 catalog_manager.cc:5495] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e reported cstate change: term changed from 0 to 1, leader changed from <none> to e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129). New cstate: current_term: 1 leader_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7bfe365d3904531b2ce0c5d0577f09e" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 45623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 } health_report { overall_health: UNKNOWN } } }
W20260807 18:10:09.576143 25570 tablet.cc:2369] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:09.583526 25296 tablet.cc:2369] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:09.590063 25437 tablet.cc:2369] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:10:09.603091 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.597298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49870 (local address 127.8.62.129:45623)
0807 18:10:09.597551 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.597558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.597669 (+ 111us) server_negotiation.cc:407] Connection header received
0807 18:10:09.597929 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.597933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.598027 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.598188 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:10:09.598567 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.599338 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.600791 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.601218 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.601357 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.601931 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:10:09.602478 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:10:09.602664 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.602719 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.602809 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:10:09.613276 25524 raft_consensus.cc:1261] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Refusing update from remote peer e7bfe365d3904531b2ce0c5d0577f09e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:09.613323 25388 raft_consensus.cc:1261] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Refusing update from remote peer e7bfe365d3904531b2ce0c5d0577f09e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:09.614552 25593 consensus_queue.cc:1035] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a86721240cb74c33a98e603f46a70179" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:09.614914 25597 consensus_queue.cc:1035] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f201064af0354bf4a13d94e9de10ec85" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:09.627347 25601 mvcc.cc:205] Tried to move back new op lower bound from 7315972954563616768 to 7315972954374586368. Current Snapshot: MvccSnapshot[applied={T|T < 7315972954563616768}]
I20260807 18:10:09.628259 25600 mvcc.cc:205] Tried to move back new op lower bound from 7315972954563616768 to 7315972954374586368. Current Snapshot: MvccSnapshot[applied={T|T < 7315972954563616768}]
I20260807 18:10:09.645262 25102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:09.645641 25102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:09.650324 25102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Sending DeleteTablet for 3 replicas of tablet a274a180e0be41b79354c85c0f6189a0
I20260807 18:10:09.651865 25503 tablet_service.cc:1514] Processing DeleteTablet for tablet a274a180e0be41b79354c85c0f6189a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:09 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42238
I20260807 18:10:09.651877 25228 tablet_service.cc:1514] Processing DeleteTablet for tablet a274a180e0be41b79354c85c0f6189a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:09 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:49858
I20260807 18:10:09.651999 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25141
I20260807 18:10:09.653841 25368 tablet_service.cc:1514] Processing DeleteTablet for tablet a274a180e0be41b79354c85c0f6189a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:09 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:33050
I20260807 18:10:09.668076 25612 tablet_replica.cc:330] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: stopping tablet replica
I20260807 18:10:09.668939 25612 raft_consensus.cc:2229] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:10:09.669813 25612 raft_consensus.cc:2258] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:09.672412 25612 ts_tablet_manager.cc:1905] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:09.681452 25612 ts_tablet_manager.cc:1918] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:09.681712 25612 log.cc:1196] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/a274a180e0be41b79354c85c0f6189a0
I20260807 18:10:09.682412 25612 ts_tablet_manager.cc:1939] T a274a180e0be41b79354c85c0f6189a0 P e7bfe365d3904531b2ce0c5d0577f09e: Deleting consensus metadata
I20260807 18:10:09.684737 25087 catalog_manager.cc:4841] TS e7bfe365d3904531b2ce0c5d0577f09e (127.8.62.129:45623): tablet a274a180e0be41b79354c85c0f6189a0 (table test-table [id=559ada67fb804460b16d5dd77320c664]) successfully deleted
I20260807 18:10:09.686337 25613 tablet_replica.cc:330] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: stopping tablet replica
I20260807 18:10:09.687004 25613 raft_consensus.cc:2229] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:09.687505 25584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.654828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49872 (local address 127.8.62.129:45623)
0807 18:10:09.655326 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.655335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.655360 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:10:09.655446 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.655454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.655572 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.655734 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.658683 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.660592 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.664080 (+ 3488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.666441 (+ 2361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.666975 (+ 534us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.675504 (+ 8529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.675522 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.675541 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.675607 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.680397 (+ 4790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.682981 (+ 2584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.682994 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.682997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.683217 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.684157 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.684162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.684165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.685276 (+ 1111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.685649 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.686821 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.687108 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
I20260807 18:10:09.688359 25613 raft_consensus.cc:2258] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:09.688485 25290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:09.691363 25613 ts_tablet_manager.cc:1905] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:09.696686 25613 ts_tablet_manager.cc:1918] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:09.696882 25613 log.cc:1196] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/a274a180e0be41b79354c85c0f6189a0
I20260807 18:10:09.697495 25611 tablet_replica.cc:330] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: stopping tablet replica
I20260807 18:10:09.697515 25613 ts_tablet_manager.cc:1939] T a274a180e0be41b79354c85c0f6189a0 P f201064af0354bf4a13d94e9de10ec85: Deleting consensus metadata
I20260807 18:10:09.697991 25611 raft_consensus.cc:2229] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:09.698544 25611 raft_consensus.cc:2258] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:09.699533 25088 catalog_manager.cc:4841] TS f201064af0354bf4a13d94e9de10ec85 (127.8.62.130:42689): tablet a274a180e0be41b79354c85c0f6189a0 (table test-table [id=559ada67fb804460b16d5dd77320c664]) successfully deleted
I20260807 18:10:09.700708 25611 ts_tablet_manager.cc:1905] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:09.705651 25611 ts_tablet_manager.cc:1918] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:09.705821 25611 log.cc:1196] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/a274a180e0be41b79354c85c0f6189a0
I20260807 18:10:09.706354 25611 ts_tablet_manager.cc:1939] T a274a180e0be41b79354c85c0f6189a0 P a86721240cb74c33a98e603f46a70179: Deleting consensus metadata
I20260807 18:10:09.708048 25089 catalog_manager.cc:4841] TS a86721240cb74c33a98e603f46a70179 (127.8.62.131:35405): tablet a274a180e0be41b79354c85c0f6189a0 (table test-table [id=559ada67fb804460b16d5dd77320c664]) successfully deleted
I20260807 18:10:09.793955 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25141
I20260807 18:10:09.816154 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25299
I20260807 18:10:09.834496 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.817106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.8.62.130:42689)
0807 18:10:09.817281 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.817286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.817305 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:10:09.817382 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.817386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.817449 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.817539 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.819062 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.819850 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.821277 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.821701 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.821800 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.828846 (+ 7046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.828865 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.828882 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.828928 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.831828 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.832502 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.832508 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.832511 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.832700 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.833027 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.833030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.833032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.833410 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.833646 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.833912 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.834105 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 18:10:09.835216 25431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:09.927341 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25299
I20260807 18:10:09.945566 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25439
I20260807 18:10:09.960834 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:09.946444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.8.62.131:35405)
0807 18:10:09.946646 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:09.946650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:09.946671 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:10:09.946827 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:09.946832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:09.946917 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:09.947006 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:09.948710 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.949445 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.950886 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:09.951289 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:09.951388 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:09.954491 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:09.954507 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:09.954524 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:09.954561 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:09.957584 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.958404 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.958413 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.958415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.958582 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:09.959067 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:09.959071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:09.959073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:09.959472 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:09.959687 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:09.960009 (+ 322us) server_negotiation.cc:299] Negotiation successful
0807 18:10:09.960206 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:10:09.961519 25564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:10.052429 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25439
I20260807 18:10:10.071432 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25069
I20260807 18:10:10.072795 25134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:10.190625 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25069
I20260807 18:10:30.216233 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--webserver_interface=127.8.62.190
--webserver_port=41415
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:43875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:30.571429 25618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:30.571755 25618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:30.571928 25618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:30.579355 25618 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 18:10:30.579463 25618 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 18:10:30.579532 25618 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 18:10:30.579612 25618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:10:30.579695 25618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:30.579807 25618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:30.579864 25618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:10:30.579942 25618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:10:30.590608 25618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:43875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:43875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=41415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:30.592024 25618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:30.593425 25618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:30.606158 25624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:30.606072 25623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:30.606191 25626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:30.607183 25618 server_base.cc:1034] running on GCE node
I20260807 18:10:30.608088 25618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:30.609727 25618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:30.610978 25618 hybrid_clock.cc:648] HybridClock initialized: now 1786126230610896 us; error 62 us; skew 500 ppm
I20260807 18:10:30.611330 25618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:30 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126230, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:30.622437 25618 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:10:30.624250 25618 webserver.cc:460] Webserver started at http://127.8.62.190:41415/ using document root <none> and password file <none>
I20260807 18:10:30.625134 25618 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:30.625277 25618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:30.631102 25618 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260807 18:10:30.634217 25634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:30.635707 25618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:10:30.635906 25618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "55b5965c6aae4f5f8bb6ef8369b22d75"
format_stamp: "Formatted at 2026-08-07 18:09:44 on dist-test-slave-rnrw"
server_key: "176f5f215a280f85ae29c469a2b8f7fd"
server_key_iv: "3339d3e80c31ec2e09fad16d1993eae9"
server_key_version: "encryptionkey@0"
I20260807 18:10:30.637027 25618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:30.651839 25618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:30.656350 25618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:30.656697 25618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:30.677541 25618 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:43875
I20260807 18:10:30.677635 25688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:43875 every 8 connection(s)
I20260807 18:10:30.679147 25618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:10:30.684047 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25618
I20260807 18:10:30.685256 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:45623
--local_ip_for_outbound_sockets=127.8.62.129
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=37125
--webserver_interface=127.8.62.129
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:10:30.688337 25689 sys_catalog.cc:263] Verifying existing consensus state
I20260807 18:10:30.694381 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap starting.
I20260807 18:10:30.718641 25689 log.cc:824] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:30.736897 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 18:10:30.737556 25689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Bootstrap complete.
I20260807 18:10:30.748622 25689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:30.749660 25689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Initialized, Role: FOLLOWER
I20260807 18:10:30.750365 25689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:30.750687 25689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:10:30.750856 25689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:10:30.751080 25689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 2 FOLLOWER]: Advancing to term 3
I20260807 18:10:30.755391 25689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:30.755959 25689 leader_election.cc:302] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55b5965c6aae4f5f8bb6ef8369b22d75; no voters:
I20260807 18:10:30.756727 25689 leader_election.cc:288] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 18:10:30.756886 25693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 18:10:30.758203 25693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [term 3 LEADER]: Becoming Leader. State: Replica: 55b5965c6aae4f5f8bb6ef8369b22d75, State: Running, Role: LEADER
I20260807 18:10:30.758829 25693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } }
I20260807 18:10:30.759213 25689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:10:30.761581 25696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55b5965c6aae4f5f8bb6ef8369b22d75. Latest consensus state: current_term: 3 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:10:30.761973 25696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:30.762532 25695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55b5965c6aae4f5f8bb6ef8369b22d75" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 43875 } } }
I20260807 18:10:30.762823 25695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:30.767526 25700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:10:30.780120 25700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:10:30.783136 25700 catalog_manager.cc:1259] Loaded cluster ID: 8dcdf638728c4bc29386f78203d99707
I20260807 18:10:30.783269 25700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:10:30.789785 25700 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:10:30.790668 25711 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:10:30.791934 25700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Loaded TSK: 0
I20260807 18:10:30.792119 25700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 55b5965c6aae4f5f8bb6ef8369b22d75: Loaded TSK: 1
F20260807 18:10:30.793170 25700 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5b5fd5d9ad google::LogMessage::Fail() at ??:0
@ 0x7f5b5fd6088a google::LogMessage::SendToLog() at ??:0
@ 0x7f5b5fd5d4de google::LogMessage::Flush() at ??:0
@ 0x7f5b5fd5ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5b63ac17d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5b63ab073e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5b7578c1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5b7578b783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5b758172a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f5b758170c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f5b758615e9 std::function<>::operator()() at ??:0
@ 0x7f5b7578fcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5b7578dc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5b75815610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5b75815439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b629a9345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5b629ac91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5b629ac799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5b62987eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5b65c616db start_thread at ??:0
@ 0x7f5b5ebf761f clone at ??:0
W20260807 18:10:31.087595 25691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:31.088052 25691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:31.088265 25691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:31.102988 25691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:31.103124 25691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:31.103250 25691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:10:31.113946 25691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=37125
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:31.115679 25691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:31.117121 25691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:31.131431 25738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:31.131438 25739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:31.131745 25741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:31.132200 25691 server_base.cc:1034] running on GCE node
I20260807 18:10:31.132886 25691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:31.133880 25691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:31.135113 25691 hybrid_clock.cc:648] HybridClock initialized: now 1786126231135053 us; error 48 us; skew 500 ppm
I20260807 18:10:31.135583 25691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:31 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126231, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:31.139292 25691 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:10:31.141026 25691 webserver.cc:460] Webserver started at http://127.8.62.129:37125/ using document root <none> and password file <none>
I20260807 18:10:31.141806 25691 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:31.141924 25691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:31.148092 25691 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:10:31.151491 25748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.153147 25691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:10:31.153319 25691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "e7bfe365d3904531b2ce0c5d0577f09e"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "acfbdf31457c08292df1e27a7607bbcd"
server_key_iv: "6220de5fa1e508e74edf781a6a2e91cf"
server_key_version: "encryptionkey@0"
I20260807 18:10:31.154510 25691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:31.194172 25691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:31.198732 25691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:31.199036 25691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:31.200131 25691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:31.201555 25691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:31.201699 25691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.201834 25691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:31.201924 25691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.237665 25691 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:45623
I20260807 18:10:31.237761 25864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:45623 every 8 connection(s)
I20260807 18:10:31.239223 25691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:10:31.244287 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.242385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:55915)
0807 18:10:31.243174 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.243400 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":314,"threads_started":1}
W20260807 18:10:31.245018 25865 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:43875 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:43875: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
I20260807 18:10:31.245872 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.245543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:53987)
0807 18:10:31.245705 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.245772 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 18:10:31.246830 25867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.246498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:34231)
0807 18:10:31.246654 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.246727 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 18:10:31.246901 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25691
W20260807 18:10:31.247169 25865 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:10:31.247648 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:42689
--local_ip_for_outbound_sockets=127.8.62.130
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=44199
--webserver_interface=127.8.62.130
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:31.596114 25868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:31.596403 25868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:31.596511 25868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:31.604612 25868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:31.604733 25868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:31.604848 25868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:10:31.614811 25868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:42689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=44199
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:31.616094 25868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:31.617412 25868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:31.630968 25873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:31.630962 25874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:31.631171 25876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:31.632088 25868 server_base.cc:1034] running on GCE node
I20260807 18:10:31.632802 25868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:31.633822 25868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:31.635076 25868 hybrid_clock.cc:648] HybridClock initialized: now 1786126231634997 us; error 55 us; skew 500 ppm
I20260807 18:10:31.635499 25868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:31 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126231, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:31.638993 25868 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:10:31.640648 25868 webserver.cc:460] Webserver started at http://127.8.62.130:44199/ using document root <none> and password file <none>
I20260807 18:10:31.641389 25868 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:31.641520 25868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:31.647264 25868 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 18:10:31.650360 25883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.651986 25868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:10:31.652170 25868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "f201064af0354bf4a13d94e9de10ec85"
format_stamp: "Formatted at 2026-08-07 18:09:45 on dist-test-slave-rnrw"
server_key: "e87872e0a3dac980e7d582c63ff51aa5"
server_key_iv: "dbe549382b05c6a06a5aaa218cdafdcb"
server_key_version: "encryptionkey@0"
I20260807 18:10:31.653273 25868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:31.678076 25868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:31.682504 25868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:31.682811 25868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:31.683867 25868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:31.685323 25868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:31.685458 25868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.685595 25868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:31.685693 25868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:31.720657 25868 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:42689
I20260807 18:10:31.720782 26001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:42689 every 8 connection(s)
I20260807 18:10:31.722148 25868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:10:31.724040 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 25868
I20260807 18:10:31.724817 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:35405
--local_ip_for_outbound_sockets=127.8.62.131
--tserver_master_addrs=127.8.62.190:43875
--webserver_port=34491
--webserver_interface=127.8.62.131
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:10:31.727609 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.725537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:57223)
0807 18:10:31.726383 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.726658 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":350,"threads_started":1}
W20260807 18:10:31.728442 26002 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:43875 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:43875: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
I20260807 18:10:31.729347 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.729011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:49261)
0807 18:10:31.729169 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.729247 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 18:10:31.730180 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:31.729852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:44291)
0807 18:10:31.729996 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:31.730075 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 18:10:31.730399 26002 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 18:10:32.061163 26005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:32.061487 26005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:32.061656 26005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:32.069120 26005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:32.069279 26005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:32.069449 26005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:10:32.079380 26005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:35405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=34491
--tserver_master_addrs=127.8.62.190:43875
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:32.080816 26005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:32.082135 26005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:32.096304 26011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:32.096374 26013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:32.096374 26010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:32.097309 26005 server_base.cc:1034] running on GCE node
I20260807 18:10:32.098040 26005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:32.099050 26005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:32.100308 26005 hybrid_clock.cc:648] HybridClock initialized: now 1786126232100250 us; error 51 us; skew 500 ppm
I20260807 18:10:32.100742 26005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:32 dist-test-slave-rnrw krb5kdc[24494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126232, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:32.104044 26005 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:10:32.105628 26005 webserver.cc:460] Webserver started at http://127.8.62.131:34491/ using document root <none> and password file <none>
I20260807 18:10:32.106374 26005 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:32.106526 26005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:32.112318 26005 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 18:10:32.115228 26020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:32.116765 26005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:10:32.116973 26005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "a86721240cb74c33a98e603f46a70179"
format_stamp: "Formatted at 2026-08-07 18:09:46 on dist-test-slave-rnrw"
server_key: "de1ec72e87bbd97b0a82453e4734dc50"
server_key_iv: "e55e8f2c6c93f64149145a3daef150b2"
server_key_version: "encryptionkey@0"
I20260807 18:10:32.117986 26005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:32.133138 26005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:32.137542 26005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:32.137843 26005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:32.138867 26005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:32.140234 26005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:32.140368 26005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:32.140511 26005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:32.140617 26005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260807 18:10:32.176755 26005 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:35405
I20260807 18:10:32.176848 26139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:35405 every 8 connection(s)
I20260807 18:10:32.178375 26005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:10:32.181878 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26005
I20260807 18:10:32.184921 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:32.181866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:43363)
0807 18:10:32.183364 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:32.183641 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":765,"threads_started":1}
W20260807 18:10:32.185736 26140 heartbeater.cc:644] Failed to heartbeat to 127.8.62.190:43875 (0 consecutive failures): Network error: Failed to ping master at 127.8.62.190:43875: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
I20260807 18:10:32.186653 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:32.186296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:45757)
0807 18:10:32.186459 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:32.186539 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 18:10:32.187760 26142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:32.187246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:38975)
0807 18:10:32.187557 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:32.187644 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
W20260807 18:10:32.188033 26140 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 18:10:32.249043 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:32.248045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:47727)
0807 18:10:32.248659 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:32.248780 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":264,"threads_started":1}
I20260807 18:10:32.731810 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:32.730964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:50857)
0807 18:10:32.731436 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:32.731568 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":222,"threads_started":1}
I20260807 18:10:33.189661 26146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:33.188776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:51841)
0807 18:10:33.189290 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:33.189402 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":280,"threads_started":1}
I20260807 18:10:33.250751 26147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:33.249960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:53307)
0807 18:10:33.250444 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:33.250567 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260807 18:10:33.733709 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:33.732854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:56721)
0807 18:10:33.733358 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:33.733444 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":256,"threads_started":1}
I20260807 18:10:34.191507 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:34.190687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:43171)
0807 18:10:34.191229 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:34.191321 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":252,"threads_started":1}
I20260807 18:10:34.252394 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:34.251677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:49191)
0807 18:10:34.252137 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:34.252218 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":226,"threads_started":1}
I20260807 18:10:34.735312 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:34.734617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:53733)
0807 18:10:34.735051 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:34.735129 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":221,"threads_started":1}
I20260807 18:10:35.193439 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:35.192719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:44957)
0807 18:10:35.193175 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:35.193262 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":238,"threads_started":1}
I20260807 18:10:35.254163 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:35.253402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:51965)
0807 18:10:35.253882 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:35.253964 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260807 18:10:35.737154 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:35.736280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:43123)
0807 18:10:35.736894 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:35.736976 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":341,"threads_started":1}
I20260807 18:10:36.195330 26155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:36.194521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:50449)
0807 18:10:36.195028 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:36.195119 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
I20260807 18:10:36.255908 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:36.255164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:50457)
0807 18:10:36.255651 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:36.255735 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":244,"threads_started":1}
I20260807 18:10:36.738839 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:36.738072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:37483)
0807 18:10:36.738572 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:36.738665 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":249,"threads_started":1}
I20260807 18:10:37.197239 26158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:37.196398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:39473)
0807 18:10:37.196967 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:37.197053 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":329,"threads_started":1}
I20260807 18:10:37.257668 26159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:37.256848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:54935)
0807 18:10:37.257339 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:37.257427 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":265,"threads_started":1}
I20260807 18:10:37.740994 26160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:37.739865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:45455)
0807 18:10:37.740697 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:37.740807 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":294,"threads_started":1}
I20260807 18:10:38.199151 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:38.198289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:40943)
0807 18:10:38.198857 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:38.198972 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":308,"threads_started":1}
I20260807 18:10:38.259629 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:38.258671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:42943)
0807 18:10:38.259272 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:38.259356 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":262,"threads_started":1}
I20260807 18:10:38.743196 26163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:38.742258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:44207)
0807 18:10:38.742855 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:38.742957 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":305,"threads_started":1}
I20260807 18:10:39.201289 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:39.200246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:59475)
0807 18:10:39.200983 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:39.201068 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":438,"threads_started":1}
I20260807 18:10:39.261547 26165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:39.260762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:44185)
0807 18:10:39.261292 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:39.261379 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":264,"threads_started":1}
I20260807 18:10:39.745203 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:39.744371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:54583)
0807 18:10:39.744937 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:39.745029 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":340,"threads_started":1}
I20260807 18:10:40.203202 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:40.202378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:42643)
0807 18:10:40.202924 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:40.203018 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":301,"threads_started":1}
I20260807 18:10:40.263252 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:40.262439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:52021)
0807 18:10:40.262963 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:40.263072 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":274,"threads_started":1}
I20260807 18:10:40.747071 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:40.746217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:45873)
0807 18:10:40.746784 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:40.746878 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":250,"threads_started":1}
I20260807 18:10:41.205314 26171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:41.204365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:59917)
0807 18:10:41.205023 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:41.205124 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":359,"threads_started":1}
I20260807 18:10:41.265336 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:41.264360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:35893)
0807 18:10:41.265004 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:41.265097 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":371,"threads_started":1}
I20260807 18:10:41.749022 26173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:41.748159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:40803)
0807 18:10:41.748759 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:41.748848 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":248,"threads_started":1}
I20260807 18:10:42.207003 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:42.206265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:58929)
0807 18:10:42.206749 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:42.206833 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":278,"threads_started":1}
I20260807 18:10:42.267076 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:42.266262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:39401)
0807 18:10:42.266781 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:42.266892 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":269,"threads_started":1}
I20260807 18:10:42.751171 26176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:42.750092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:56117)
0807 18:10:42.750893 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:42.750983 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":584,"threads_started":1}
I20260807 18:10:43.209050 26177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:43.208053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:37345)
0807 18:10:43.208669 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:43.208802 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":288,"threads_started":1}
I20260807 18:10:43.268949 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:43.268106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:55523)
0807 18:10:43.268692 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:43.268783 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":250,"threads_started":1}
I20260807 18:10:43.752887 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:43.752086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:60663)
0807 18:10:43.752626 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:43.752713 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260807 18:10:44.210803 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:44.210069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:57817)
0807 18:10:44.210532 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:44.210641 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":242,"threads_started":1}
I20260807 18:10:44.270835 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:44.270008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:50069)
0807 18:10:44.270509 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:44.270637 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":249,"threads_started":1}
I20260807 18:10:44.754667 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:44.753853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:49489)
0807 18:10:44.754352 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:44.754448 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":244,"threads_started":1}
I20260807 18:10:45.212714 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:45.211855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:56925)
0807 18:10:45.212370 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:45.212471 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":264,"threads_started":1}
I20260807 18:10:45.272531 26184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:45.271775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:33887)
0807 18:10:45.272259 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:45.272343 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":253,"threads_started":1}
I20260807 18:10:45.756283 26185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:45.755554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:33749)
0807 18:10:45.756016 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:45.756099 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260807 18:10:46.214413 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:46.213700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:36345)
0807 18:10:46.214134 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:46.214234 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260807 18:10:46.274705 26187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:46.273974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.129:35827)
0807 18:10:46.274433 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:46.274521 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":227,"threads_started":1}
I20260807 18:10:46.758435 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:46.757508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.130:54281)
0807 18:10:46.758070 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:46.758193 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":326,"threads_started":1}
I20260807 18:10:47.193136 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25691
I20260807 18:10:47.217125 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:47.215797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:43875 (local address 127.8.62.131:57465)
0807 18:10:47.216745 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:47.216853 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.190:43875: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":802,"thread_start_us":671,"threads_started":1}
I20260807 18:10:47.219223 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:47.194595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.8.62.129:45623)
0807 18:10:47.195437 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:47.195459 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:47.195509 (+ 50us) server_negotiation.cc:407] Connection header received
0807 18:10:47.197259 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:47.197269 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:47.197749 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:47.198129 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:47.199741 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.201093 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.202390 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.203091 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.203303 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:47.206898 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:47.206909 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:47.206936 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:47.207464 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:47.214579 (+ 7115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.215631 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.215645 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.215657 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.215895 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.216400 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.216409 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.216415 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.217151 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:47.218354 (+ 1203us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:47.218583 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 18:10:47.218930 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":449,"threads_started":1}
I20260807 18:10:47.220983 25860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:47.366262 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25691
I20260807 18:10:47.392678 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 25868
I20260807 18:10:47.412413 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:47.393724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.8.62.130:42689)
0807 18:10:47.394343 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:47.394356 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:47.394396 (+ 40us) server_negotiation.cc:407] Connection header received
0807 18:10:47.396052 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:47.396064 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:47.396509 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:47.396961 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:47.398467 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.399800 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.401306 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.402045 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.402275 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:47.405124 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:47.405133 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:47.405149 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:47.405188 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:47.408874 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.409624 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.409637 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.409644 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.409821 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.410242 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.410248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.410251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.410812 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:47.411149 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:47.411642 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 18:10:47.412084 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":351,"threads_started":1}
I20260807 18:10:47.414130 25997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:47.500066 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 25868
I20260807 18:10:47.522326 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26005
I20260807 18:10:47.542150 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:47.523703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50718 (local address 127.8.62.131:35405)
0807 18:10:47.524404 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:47.524418 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:47.524461 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:10:47.526174 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:47.526184 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:47.526624 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:47.526923 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:47.528407 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.529742 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.531051 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:47.531826 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:47.531994 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:47.535187 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:47.535196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:47.535212 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:47.535254 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:47.538435 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.539189 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.539196 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.539203 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.539377 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:47.539814 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:47.539818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:47.539820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:47.540355 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:47.540681 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:47.541381 (+ 700us) server_negotiation.cc:299] Negotiation successful
0807 18:10:47.541847 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":378,"threads_started":1}
I20260807 18:10:47.543574 26135 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:47.628993 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26005
2026-08-07T18:10:47Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65010 ms)
[----------] 30 tests from SecurityITest (314582 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:10:47 dist-test-slave-rnrw krb5kdc[26199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:10:47 dist-test-slave-rnrw krb5kdc[26199](info): set up 2 sockets
Aug 07 18:10:47 dist-test-slave-rnrw krb5kdc[26199](info): commencing operation
krb5kdc: starting...
W20260807 18:10:48.771919 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:10:48 dist-test-slave-rnrw krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:10:49.060784 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:42243
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:42243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:49.415040 26212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:49.415431 26212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:49.415618 26212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:49.423204 26212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:10:49.423401 26212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:49.423516 26212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:49.423622 26212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:10:49.423704 26212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:10:49.434173 26212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:42243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:42243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:49.435746 26212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:49.437201 26212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:49.450994 26217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:49.451084 26218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:49.451105 26220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:49.451376 26212 server_base.cc:1034] running on GCE node
I20260807 18:10:49.452589 26212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:49.454129 26212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:49.455354 26212 hybrid_clock.cc:648] HybridClock initialized: now 1786126249455282 us; error 55 us; skew 500 ppm
I20260807 18:10:49.455807 26212 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:49 dist-test-slave-rnrw krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126249, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:49.464000 26212 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:10:49.465677 26212 webserver.cc:460] Webserver started at http://127.8.62.190:45469/ using document root <none> and password file <none>
I20260807 18:10:49.466378 26212 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:49.466495 26212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:49.466766 26212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:49.470045 26212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "7761a9183ad14a20a503844eb839ca3e"
format_stamp: "Formatted at 2026-08-07 18:10:49 on dist-test-slave-rnrw"
server_key: "4c02c087a68115413e383da7a4ccfa72"
server_key_iv: "1ed74861c1796bd29f3da9600d2b1b4d"
server_key_version: "encryptionkey@0"
I20260807 18:10:49.470829 26212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "7761a9183ad14a20a503844eb839ca3e"
format_stamp: "Formatted at 2026-08-07 18:10:49 on dist-test-slave-rnrw"
server_key: "4c02c087a68115413e383da7a4ccfa72"
server_key_iv: "1ed74861c1796bd29f3da9600d2b1b4d"
server_key_version: "encryptionkey@0"
I20260807 18:10:49.476603 26212 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:10:49.480474 26227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:49.482049 26212 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 18:10:49.482296 26212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "7761a9183ad14a20a503844eb839ca3e"
format_stamp: "Formatted at 2026-08-07 18:10:49 on dist-test-slave-rnrw"
server_key: "4c02c087a68115413e383da7a4ccfa72"
server_key_iv: "1ed74861c1796bd29f3da9600d2b1b4d"
server_key_version: "encryptionkey@0"
I20260807 18:10:49.482534 26212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:49.506435 26212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:49.511053 26212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:49.511386 26212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:49.531230 26212 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:42243
I20260807 18:10:49.531281 26280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:42243 every 8 connection(s)
I20260807 18:10:49.532734 26212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:10:49.537307 26281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:49.540088 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26212
I20260807 18:10:49.540316 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:10:49.541137 8442 external_mini_cluster.cc:1428] Setting key 6628eaad8cab3f6b1412178d8ee6d058
I20260807 18:10:49.550388 26281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: Bootstrap starting.
I20260807 18:10:49.554000 26281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:49.555109 26281 log.cc:824] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: Log is configured to *not* fsync() on all Append() calls
Aug 07 18:10:49 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:49.558629 26281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: No bootstrap required, opened a new log
I20260807 18:10:49.565864 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:49.543526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.8.62.190:42243)
0807 18:10:49.544407 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:49.544420 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:49.544462 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:10:49.546280 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:49.546303 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:49.546753 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:49.547208 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:49.548853 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:49.550205 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:49.551807 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:49.552597 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:49.552807 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:49.557567 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:49.557575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:49.557593 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:49.557636 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:49.561317 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:49.562249 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:49.562256 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:49.562262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:49.562448 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:49.562881 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:49.562884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:49.562886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:49.563461 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:49.563713 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:49.564239 (+ 526us) server_negotiation.cc:299] Negotiation successful
0807 18:10:49.564730 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":409,"threads_started":1}
I20260807 18:10:49.568032 26281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } }
I20260807 18:10:49.568463 26281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:49.568624 26281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7761a9183ad14a20a503844eb839ca3e, State: Initialized, Role: FOLLOWER
I20260807 18:10:49.569260 26281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } }
I20260807 18:10:49.569512 26281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:10:49.569685 26281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:10:49.569865 26281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:49.572399 26281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } }
I20260807 18:10:49.572991 26281 leader_election.cc:302] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7761a9183ad14a20a503844eb839ca3e; no voters:
I20260807 18:10:49.573655 26281 leader_election.cc:288] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:10:49.573834 26286 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:49.575112 26286 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [term 1 LEADER]: Becoming Leader. State: Replica: 7761a9183ad14a20a503844eb839ca3e, State: Running, Role: LEADER
I20260807 18:10:49.575681 26286 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } }
I20260807 18:10:49.575963 26281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:10:49.579020 26288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7761a9183ad14a20a503844eb839ca3e. Latest consensus state: current_term: 1 leader_uuid: "7761a9183ad14a20a503844eb839ca3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } } }
I20260807 18:10:49.579698 26288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:49.579070 26287 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7761a9183ad14a20a503844eb839ca3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7761a9183ad14a20a503844eb839ca3e" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 42243 } } }
I20260807 18:10:49.580341 26287 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:49.584183 26295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:10:49.588981 26295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:10:49.598001 26295 catalog_manager.cc:1347] Generated new cluster ID: 58a841e1ae574e01a2d72b4fecdfcc6d
I20260807 18:10:49.598155 26295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:10:49.622044 26295 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:10:49.623443 26295 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:10:49.634172 26295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: Generated new TSK 0
I20260807 18:10:49.635104 26295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:10:49.707854 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:50.061621 26309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:50.061911 26309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:50.062069 26309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:50.069577 26309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:50.069701 26309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:50.069892 26309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:10:50.080322 26309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:50.081749 26309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:50.083124 26309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:50.097102 26314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:50.097102 26315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:50.097385 26317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:50.098461 26309 server_base.cc:1034] running on GCE node
I20260807 18:10:50.099058 26309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:50.100010 26309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:50.101210 26309 hybrid_clock.cc:648] HybridClock initialized: now 1786126250101140 us; error 63 us; skew 500 ppm
I20260807 18:10:50.101503 26309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:50 dist-test-slave-rnrw krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:50.104883 26309 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:10:50.106463 26309 webserver.cc:460] Webserver started at http://127.8.62.129:36177/ using document root <none> and password file <none>
I20260807 18:10:50.107141 26309 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:50.107249 26309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:50.107539 26309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:50.110570 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "463d55f653c747f0ae38f174c62a8524"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "96fc8f4b36099ee1f21dd39c1108b37b"
server_key_iv: "4a52d340748d3b0a5d8319beec04bc65"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.111351 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "463d55f653c747f0ae38f174c62a8524"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "96fc8f4b36099ee1f21dd39c1108b37b"
server_key_iv: "4a52d340748d3b0a5d8319beec04bc65"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.116757 26309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:10:50.120358 26324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.121867 26309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:10:50.122025 26309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "463d55f653c747f0ae38f174c62a8524"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "96fc8f4b36099ee1f21dd39c1108b37b"
server_key_iv: "4a52d340748d3b0a5d8319beec04bc65"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.122236 26309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:50.144671 26309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:50.149719 26309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:50.150066 26309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:50.151129 26309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:50.152650 26309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:50.152732 26309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.152921 26309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:50.152979 26309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.189164 26309 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:34283
I20260807 18:10:50.189266 26441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:34283 every 8 connection(s)
I20260807 18:10:50.190605 26309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:10:50.196265 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26309
I20260807 18:10:50.196501 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:10:50.197384 8442 external_mini_cluster.cc:1428] Setting key bcd6a5611c23b4cbd837f9b63b229951
Aug 07 18:10:50 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:50.213228 26444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:50.193433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:47819 (local address 127.8.62.190:42243)
0807 18:10:50.193899 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:50.193904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:50.195071 (+ 1167us) server_negotiation.cc:407] Connection header received
0807 18:10:50.197159 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:50.197167 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:50.197288 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:50.197440 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:50.199199 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.199951 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.201680 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.202098 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.202242 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:50.205990 (+ 3748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:50.205996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:50.205999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:50.206034 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:50.209371 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:50.210279 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:50.210288 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:50.210294 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:50.210470 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:50.210995 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:50.211003 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:50.211009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:50.211406 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:50.211574 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:50.212375 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 18:10:50.212710 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":245,"threads_started":1}
I20260807 18:10:50.213999 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:50.194118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:42243 (local address 127.8.62.129:47819)
0807 18:10:50.194878 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:50.194928 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:50.196833 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:50.197752 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:50.197761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:50.198381 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:50.198999 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:50.199018 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.200137 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.200142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:50.201514 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:50.201527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.202356 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.202364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:50.202614 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:50.203342 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:50.203367 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:50.205715 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:50.209579 (+ 3864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:50.209588 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:50.209603 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:50.210125 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:50.210635 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:50.210639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:50.210641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:50.210860 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:50.211548 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:50.211555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:50.211928 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 18:10:50.212184 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 18:10:50.212761 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":291,"threads_started":1}
I20260807 18:10:50.216042 26442 heartbeater.cc:347] Connected to a master server at 127.8.62.190:42243
I20260807 18:10:50.216431 26442 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:50.217507 26442 heartbeater.cc:511] Master 127.8.62.190:42243 requested a full tablet report, sending...
I20260807 18:10:50.220383 26245 ts_manager.cc:194] Registered new tserver with Master: 463d55f653c747f0ae38f174c62a8524 (127.8.62.129:34283)
I20260807 18:10:50.222828 26245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:47819
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:10:50.260289 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:50.611616 26448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:50.611951 26448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:50.612054 26448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:50.619822 26448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:50.619969 26448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:50.620088 26448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:10:50.630715 26448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:50.632138 26448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:50.633970 26448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:50.649744 26453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:50.649917 26456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:50.649917 26454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:50.650833 26448 server_base.cc:1034] running on GCE node
I20260807 18:10:50.651503 26448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:50.652510 26448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:50.653801 26448 hybrid_clock.cc:648] HybridClock initialized: now 1786126250653754 us; error 42 us; skew 500 ppm
I20260807 18:10:50.654207 26448 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:50 dist-test-slave-rnrw krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:50.657735 26448 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:10:50.659565 26448 webserver.cc:460] Webserver started at http://127.8.62.130:44821/ using document root <none> and password file <none>
I20260807 18:10:50.660352 26448 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:50.660512 26448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:50.660890 26448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:50.664026 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "5650573bd9424722b0fdc35c165c09db"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "eab60f10c555c835eff1bebc97f430d7"
server_key_iv: "ec4c8b261a1417826a3be1172b9e7394"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.664975 26448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "5650573bd9424722b0fdc35c165c09db"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "eab60f10c555c835eff1bebc97f430d7"
server_key_iv: "ec4c8b261a1417826a3be1172b9e7394"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.670817 26448 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260807 18:10:50.674829 26464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.676470 26448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:10:50.676793 26448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "5650573bd9424722b0fdc35c165c09db"
format_stamp: "Formatted at 2026-08-07 18:10:50 on dist-test-slave-rnrw"
server_key: "eab60f10c555c835eff1bebc97f430d7"
server_key_iv: "ec4c8b261a1417826a3be1172b9e7394"
server_key_version: "encryptionkey@0"
I20260807 18:10:50.677033 26448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:50.708127 26448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:50.713227 26448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:50.713548 26448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:50.714687 26448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:50.716157 26448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:50.716233 26448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.716329 26448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:50.716375 26448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:50.752694 26448 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:42991
I20260807 18:10:50.752820 26580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:42991 every 8 connection(s)
I20260807 18:10:50.754174 26448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:10:50.759318 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26448
I20260807 18:10:50.759549 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:10:50.760062 8442 external_mini_cluster.cc:1428] Setting key c09c253aef7fe21fc5db9496bdde1afd
Aug 07 18:10:50 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126250, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:50.778811 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:50.757238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:34295 (local address 127.8.62.190:42243)
0807 18:10:50.757702 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:50.757707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:50.758921 (+ 1214us) server_negotiation.cc:407] Connection header received
0807 18:10:50.761064 (+ 2143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:50.761070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:50.761180 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:50.761301 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:50.762933 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.763657 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.765368 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.765813 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.765962 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:50.770330 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:50.770337 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:50.770341 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:50.770381 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:50.773748 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:50.774960 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:50.774965 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:50.774968 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:50.775133 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:50.775715 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:50.775718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:50.775720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:50.776231 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:50.776491 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:50.777785 (+ 1294us) server_negotiation.cc:299] Negotiation successful
0807 18:10:50.778200 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":231,"threads_started":1}
I20260807 18:10:50.779994 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:50.757918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:42243 (local address 127.8.62.130:34295)
0807 18:10:50.758739 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:50.758789 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:50.760775 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:50.761564 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:50.761573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:50.762147 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:50.762753 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:50.762771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.763819 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.763823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:50.765212 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:50.765225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:50.766075 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:50.766082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:50.766329 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:50.767169 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:50.767200 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:50.770014 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:50.773975 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:50.773989 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:50.774012 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:50.774757 (+ 745us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:50.775271 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:50.775279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:50.775286 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:50.775582 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:50.776468 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:50.776508 (+ 40us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:50.777173 (+ 665us) client_negotiation.cc:769] Sending connection context
0807 18:10:50.777614 (+ 441us) client_negotiation.cc:240] Negotiation successful
0807 18:10:50.778235 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":322,"threads_started":1}
I20260807 18:10:50.784590 26581 heartbeater.cc:347] Connected to a master server at 127.8.62.190:42243
I20260807 18:10:50.785073 26581 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:50.785976 26581 heartbeater.cc:511] Master 127.8.62.190:42243 requested a full tablet report, sending...
I20260807 18:10:50.787796 26245 ts_manager.cc:194] Registered new tserver with Master: 5650573bd9424722b0fdc35c165c09db (127.8.62.130:42991)
I20260807 18:10:50.788822 26245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:34295
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:10:50.828310 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:46011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:51.182090 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:51.182473 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:51.182670 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:51.190448 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:51.190624 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:51.190822 26587 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:10:51.201336 26587 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:46011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:42243
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:51.202775 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:51.204203 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:51.218535 26595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:51.218605 26592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:51.218585 26593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:51.219431 26587 server_base.cc:1034] running on GCE node
I20260807 18:10:51.220112 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:51.221418 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:51.222688 26587 hybrid_clock.cc:648] HybridClock initialized: now 1786126251222660 us; error 74 us; skew 500 ppm
I20260807 18:10:51.223073 26587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:51 dist-test-slave-rnrw krb5kdc[26199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:51.226553 26587 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:10:51.227236 26442 heartbeater.cc:503] Master 127.8.62.190:42243 was elected leader, sending a full tablet report...
I20260807 18:10:51.228384 26587 webserver.cc:460] Webserver started at http://127.8.62.131:38793/ using document root <none> and password file <none>
I20260807 18:10:51.229285 26587 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:51.229444 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:51.229763 26587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:51.232849 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "968d79cec0b9428dbe9885c7dab4d50d"
format_stamp: "Formatted at 2026-08-07 18:10:51 on dist-test-slave-rnrw"
server_key: "5c95288517ef994f65121122f5ac77d7"
server_key_iv: "c9a9b5e537c89bd77b258767697a30c8"
server_key_version: "encryptionkey@0"
I20260807 18:10:51.233654 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "968d79cec0b9428dbe9885c7dab4d50d"
format_stamp: "Formatted at 2026-08-07 18:10:51 on dist-test-slave-rnrw"
server_key: "5c95288517ef994f65121122f5ac77d7"
server_key_iv: "c9a9b5e537c89bd77b258767697a30c8"
server_key_version: "encryptionkey@0"
I20260807 18:10:51.238941 26587 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:10:51.242938 26602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:51.244819 26587 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 18:10:51.245039 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "968d79cec0b9428dbe9885c7dab4d50d"
format_stamp: "Formatted at 2026-08-07 18:10:51 on dist-test-slave-rnrw"
server_key: "5c95288517ef994f65121122f5ac77d7"
server_key_iv: "c9a9b5e537c89bd77b258767697a30c8"
server_key_version: "encryptionkey@0"
I20260807 18:10:51.245294 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:51.261547 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:51.266145 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:51.266487 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:51.267607 26587 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:51.269232 26587 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:51.269351 26587 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:51.269526 26587 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:51.269613 26587 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:51.309257 26587 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:44483
I20260807 18:10:51.309301 26716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:44483 every 8 connection(s)
I20260807 18:10:51.310720 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:10:51.316406 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26587
I20260807 18:10:51.316664 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:10:51.317106 8442 external_mini_cluster.cc:1428] Setting key 76bf02af3dc5b3654f383b08df865dfd
Aug 07 18:10:51 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126251, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:51.333362 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.313609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:49425 (local address 127.8.62.190:42243)
0807 18:10:51.314115 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.314120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.315120 (+ 1000us) server_negotiation.cc:407] Connection header received
0807 18:10:51.317331 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.317337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.317448 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.317579 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:51.319435 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.320473 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.322277 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.322677 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.322774 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.326695 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:51.326701 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:51.326704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:51.326740 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:51.329697 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.330547 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.330551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.330553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.330681 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.331347 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.331357 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.331359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.331666 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:51.331859 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.332698 (+ 839us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.332954 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":232,"threads_started":1}
I20260807 18:10:51.334458 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.314163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:42243 (local address 127.8.62.131:49425)
0807 18:10:51.314936 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.314975 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.317009 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.317856 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.317865 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.318597 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:51.319208 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.319226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.320704 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.320709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.322125 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.322139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.322905 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.322910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.323135 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.323807 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:51.323831 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:51.326442 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:51.329870 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:51.329878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:51.329895 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:51.330394 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:51.330903 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:51.330908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:51.330912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:51.331198 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:51.331824 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:51.331833 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:51.332241 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.332542 (+ 301us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.333185 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":313,"threads_started":1}
I20260807 18:10:51.335901 26717 heartbeater.cc:347] Connected to a master server at 127.8.62.190:42243
I20260807 18:10:51.336208 26717 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:51.336984 26717 heartbeater.cc:511] Master 127.8.62.190:42243 requested a full tablet report, sending...
I20260807 18:10:51.338406 26245 ts_manager.cc:194] Registered new tserver with Master: 968d79cec0b9428dbe9885c7dab4d50d (127.8.62.131:44483)
I20260807 18:10:51.339231 26245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:49425
I20260807 18:10:51.345562 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:10:51.363860 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.350736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41678 (local address 127.8.62.190:42243)
0807 18:10:51.350905 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.350909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.351404 (+ 495us) server_negotiation.cc:407] Connection header received
0807 18:10:51.351650 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.351654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.351721 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.351809 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:51.353424 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.354170 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.355497 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.355946 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.356053 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.358534 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:51.358539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:51.358542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:51.358576 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:51.361347 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.362068 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.362072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.362074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.362201 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.362555 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.362558 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.362560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.362887 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:51.363076 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.363441 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.363624 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 18:10:51.368431 26245 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:10:51.376335 26245 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:10:51.416966 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.390566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.8.62.129:34283)
0807 18:10:51.408370 (+ 17804us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.408381 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.408418 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:10:51.408564 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.408582 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.408902 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.409266 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:51.410106 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.411524 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.414445 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.415610 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.415802 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.416051 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.416368 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.416546 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17621,"thread_start_us":17454,"threads_started":1}
I20260807 18:10:51.417063 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.390192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34283 (local address 127.0.0.1:59456)
0807 18:10:51.406704 (+ 16512us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.406761 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.406991 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.409277 (+ 2286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.409286 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.409338 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:51.409885 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.409899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.411711 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.411717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.414277 (+ 2560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.414289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.415883 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.415890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.416075 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.416133 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.416292 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.416504 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14167,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":14389,"threads_started":1}
I20260807 18:10:51.418345 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.390598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:42991 (local address 127.0.0.1:42920)
0807 18:10:51.409229 (+ 18631us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.409252 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.409380 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.410378 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.410383 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.410407 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:51.410821 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.410834 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.412731 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.412738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.415152 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.415169 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.417397 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.417404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.417617 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.417658 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.417750 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.417913 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18476,"mutex_wait_us":1,"thread_start_us":19659,"threads_started":1}
I20260807 18:10:51.419168 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.391248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.8.62.130:42991)
0807 18:10:51.408708 (+ 17460us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.408716 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.409525 (+ 809us) server_negotiation.cc:407] Connection header received
0807 18:10:51.409645 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.409655 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.409911 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.410126 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:51.411024 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.412580 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.415356 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.416864 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.417018 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.417242 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.418675 (+ 1433us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.418879 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17276,"thread_start_us":17181,"threads_started":1}
I20260807 18:10:51.419740 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.390993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:44483 (local address 127.0.0.1:40564)
0807 18:10:51.409973 (+ 18980us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.409991 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.410202 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.411065 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.411081 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.411111 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:51.411367 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.411374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.413487 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.413497 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.415583 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.415594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.418999 (+ 3405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.419004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.419154 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.419177 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.419293 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.419432 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18849,"thread_start_us":17891,"threads_started":1}
I20260807 18:10:51.420336 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.391315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.8.62.131:44483)
0807 18:10:51.407783 (+ 16468us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.407796 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.410098 (+ 2302us) server_negotiation.cc:407] Connection header received
0807 18:10:51.410408 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.410416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.410649 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.410827 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:51.411539 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.413297 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.415737 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.417536 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.418722 (+ 1186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.418890 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.419493 (+ 603us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.419689 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16260,"thread_start_us":16089,"threads_started":1}
I20260807 18:10:51.421898 26374 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf (DEFAULT_TABLE table=test-table [id=779191d32325450fbb1a81c35e38b578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:51.422825 26512 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf (DEFAULT_TABLE table=test-table [id=779191d32325450fbb1a81c35e38b578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:51.423663 26374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f52ec9de14e19bc56b551af8467bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:51.424223 26512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f52ec9de14e19bc56b551af8467bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:51.424152 26651 tablet_service.cc:1467] Processing CreateTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf (DEFAULT_TABLE table=test-table [id=779191d32325450fbb1a81c35e38b578]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:51.425678 26651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a6f52ec9de14e19bc56b551af8467bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:51.438921 26736 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Bootstrap starting.
I20260807 18:10:51.441346 26737 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Bootstrap starting.
I20260807 18:10:51.443194 26736 tablet_bootstrap.cc:654] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:51.444694 26736 log.cc:824] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:51.445365 26737 tablet_bootstrap.cc:654] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:51.446762 26737 log.cc:824] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:51.447661 26738 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Bootstrap starting.
I20260807 18:10:51.449580 26736 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: No bootstrap required, opened a new log
I20260807 18:10:51.449985 26736 ts_tablet_manager.cc:1397] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260807 18:10:51.451042 26737 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: No bootstrap required, opened a new log
I20260807 18:10:51.451308 26737 ts_tablet_manager.cc:1397] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:10:51.451661 26738 tablet_bootstrap.cc:654] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:51.452994 26738 log.cc:824] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:51.456300 26738 tablet_bootstrap.cc:492] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: No bootstrap required, opened a new log
I20260807 18:10:51.456660 26738 ts_tablet_manager.cc:1397] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260807 18:10:51.460012 26737 raft_consensus.cc:348] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.460408 26737 raft_consensus.cc:374] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:51.460525 26737 raft_consensus.cc:729] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463d55f653c747f0ae38f174c62a8524, State: Initialized, Role: FOLLOWER
I20260807 18:10:51.461215 26737 consensus_queue.cc:260] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.461898 26736 raft_consensus.cc:348] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.462363 26736 raft_consensus.cc:374] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:51.462456 26736 raft_consensus.cc:729] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5650573bd9424722b0fdc35c165c09db, State: Initialized, Role: FOLLOWER
I20260807 18:10:51.463099 26737 ts_tablet_manager.cc:1428] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260807 18:10:51.463162 26736 consensus_queue.cc:260] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.464311 26581 heartbeater.cc:503] Master 127.8.62.190:42243 was elected leader, sending a full tablet report...
I20260807 18:10:51.464915 26736 ts_tablet_manager.cc:1428] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260807 18:10:51.467581 26738 raft_consensus.cc:348] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.468099 26738 raft_consensus.cc:374] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:51.468214 26738 raft_consensus.cc:729] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 968d79cec0b9428dbe9885c7dab4d50d, State: Initialized, Role: FOLLOWER
I20260807 18:10:51.468964 26738 consensus_queue.cc:260] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.470146 26717 heartbeater.cc:503] Master 127.8.62.190:42243 was elected leader, sending a full tablet report...
I20260807 18:10:51.470839 26738 ts_tablet_manager.cc:1428] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260807 18:10:51.506549 26582 tablet.cc:2369] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:10:51.529757 26743 raft_consensus.cc:482] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:10:51.530076 26743 raft_consensus.cc:504] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.532271 26743 leader_election.cc:288] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 463d55f653c747f0ae38f174c62a8524 (127.8.62.129:34283), 968d79cec0b9428dbe9885c7dab4d50d (127.8.62.131:44483)
I20260807 18:10:51.541443 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.532921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:56273 (local address 127.8.62.129:34283)
0807 18:10:51.533182 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.533192 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.534472 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 18:10:51.534813 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.534818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.534898 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.534950 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:51.535865 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.537156 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.539583 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.540395 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.540649 (+ 254us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.540735 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.541032 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.541173 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:10:51.541558 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.532418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34283 (local address 127.8.62.130:56273)
0807 18:10:51.534339 (+ 1921us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.534363 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.534617 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.535156 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.535162 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.535205 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:51.535678 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.535691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.537326 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.537332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.539415 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.539424 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.540659 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.540665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.540857 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.540884 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.541001 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.541159 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1751,"thread_start_us":1469,"threads_started":1}
I20260807 18:10:51.542611 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.533384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:41317 (local address 127.8.62.131:44483)
0807 18:10:51.533648 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.533657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.534670 (+ 1013us) server_negotiation.cc:407] Connection header received
0807 18:10:51.534848 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.534854 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.534940 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.535049 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:51.535894 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.536720 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.539423 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.540907 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.541088 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.541166 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.542168 (+ 1002us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.542340 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:10:51.542562 26395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f52ec9de14e19bc56b551af8467bf" candidate_uuid: "5650573bd9424722b0fdc35c165c09db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463d55f653c747f0ae38f174c62a8524" is_pre_election: true
I20260807 18:10:51.543205 26395 raft_consensus.cc:2454] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5650573bd9424722b0fdc35c165c09db in term 0.
I20260807 18:10:51.543383 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.532719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:44483 (local address 127.8.62.130:41317)
0807 18:10:51.534571 (+ 1852us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:51.534588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:51.534700 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:51.535107 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:51.535112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:51.535140 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:51.535641 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.535660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.536858 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.536866 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.539198 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:51.539216 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.541348 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.541354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:51.541956 (+ 602us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.541996 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 18:10:51.542812 (+ 816us) client_negotiation.cc:240] Negotiation successful
0807 18:10:51.542995 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1731,"mutex_wait_us":26,"thread_start_us":1521,"threads_started":1}
I20260807 18:10:51.544030 26467 leader_election.cc:302] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 463d55f653c747f0ae38f174c62a8524, 5650573bd9424722b0fdc35c165c09db; no voters:
I20260807 18:10:51.544503 26743 raft_consensus.cc:2781] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:10:51.544348 26672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f52ec9de14e19bc56b551af8467bf" candidate_uuid: "5650573bd9424722b0fdc35c165c09db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968d79cec0b9428dbe9885c7dab4d50d" is_pre_election: true
I20260807 18:10:51.544724 26743 raft_consensus.cc:482] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:10:51.544857 26743 raft_consensus.cc:3037] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:51.544975 26672 raft_consensus.cc:2454] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5650573bd9424722b0fdc35c165c09db in term 0.
I20260807 18:10:51.547688 26743 raft_consensus.cc:504] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.548419 26743 leader_election.cc:288] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [CANDIDATE]: Term 1 election: Requested vote from peers 463d55f653c747f0ae38f174c62a8524 (127.8.62.129:34283), 968d79cec0b9428dbe9885c7dab4d50d (127.8.62.131:44483)
I20260807 18:10:51.548987 26395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f52ec9de14e19bc56b551af8467bf" candidate_uuid: "5650573bd9424722b0fdc35c165c09db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "463d55f653c747f0ae38f174c62a8524"
I20260807 18:10:51.548987 26672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a6f52ec9de14e19bc56b551af8467bf" candidate_uuid: "5650573bd9424722b0fdc35c165c09db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "968d79cec0b9428dbe9885c7dab4d50d"
I20260807 18:10:51.549223 26672 raft_consensus.cc:3037] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:51.549223 26395 raft_consensus.cc:3037] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:51.552726 26395 raft_consensus.cc:2454] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5650573bd9424722b0fdc35c165c09db in term 1.
I20260807 18:10:51.552724 26672 raft_consensus.cc:2454] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5650573bd9424722b0fdc35c165c09db in term 1.
I20260807 18:10:51.553232 26467 leader_election.cc:302] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 463d55f653c747f0ae38f174c62a8524, 5650573bd9424722b0fdc35c165c09db; no voters:
I20260807 18:10:51.553575 26743 raft_consensus.cc:2781] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:51.554077 26743 raft_consensus.cc:686] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 1 LEADER]: Becoming Leader. State: Replica: 5650573bd9424722b0fdc35c165c09db, State: Running, Role: LEADER
I20260807 18:10:51.554628 26743 consensus_queue.cc:237] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } }
I20260807 18:10:51.559916 26245 catalog_manager.cc:5495] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db reported cstate change: term changed from 0 to 1, leader changed from <none> to 5650573bd9424722b0fdc35c165c09db (127.8.62.130). New cstate: current_term: 1 leader_uuid: "5650573bd9424722b0fdc35c165c09db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5650573bd9424722b0fdc35c165c09db" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 42991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 } health_report { overall_health: UNKNOWN } } }
W20260807 18:10:51.562603 26718 tablet.cc:2369] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:10:51.612159 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.606491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42930 (local address 127.8.62.130:42991)
0807 18:10:51.606706 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.606710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.606729 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:10:51.606847 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.606851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.606929 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.607088 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:10:51.607517 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.608347 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.609903 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.610393 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.610505 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.611051 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:10:51.611587 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:10:51.611717 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.611821 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.611900 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:10:51.621749 26395 raft_consensus.cc:1261] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 1 FOLLOWER]: Refusing update from remote peer 5650573bd9424722b0fdc35c165c09db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:51.621749 26672 raft_consensus.cc:1261] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 1 FOLLOWER]: Refusing update from remote peer 5650573bd9424722b0fdc35c165c09db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:51.623139 26743 consensus_queue.cc:1035] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [LEADER]: Connected to new peer: Peer: permanent_uuid: "968d79cec0b9428dbe9885c7dab4d50d" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 44483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:51.623484 26748 consensus_queue.cc:1035] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [LEADER]: Connected to new peer: Peer: permanent_uuid: "463d55f653c747f0ae38f174c62a8524" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:51.637646 26752 mvcc.cc:205] Tried to move back new op lower bound from 7315973126629576704 to 7315973126372192256. Current Snapshot: MvccSnapshot[applied={T|T < 7315973126629576704}]
I20260807 18:10:51.641196 26751 mvcc.cc:205] Tried to move back new op lower bound from 7315973126629576704 to 7315973126372192256. Current Snapshot: MvccSnapshot[applied={T|T < 7315973126629576704 or (T in {7315973126629576704})}]
I20260807 18:10:51.651937 26244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:51.652287 26244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:51.656345 26244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7761a9183ad14a20a503844eb839ca3e: Sending DeleteTablet for 3 replicas of tablet 8a6f52ec9de14e19bc56b551af8467bf
I20260807 18:10:51.657693 26512 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:51 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:42920
I20260807 18:10:51.657687 26651 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:51 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:40564
I20260807 18:10:51.659150 26374 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a6f52ec9de14e19bc56b551af8467bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:51 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:59456
I20260807 18:10:51.660262 26762 tablet_replica.cc:330] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: stopping tablet replica
I20260807 18:10:51.661172 26762 raft_consensus.cc:2229] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:51.661890 26762 raft_consensus.cc:2258] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:51.663141 26760 tablet_replica.cc:330] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: stopping tablet replica
I20260807 18:10:51.663687 26760 raft_consensus.cc:2229] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:10:51.664536 26760 raft_consensus.cc:2258] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:51.665446 26761 tablet_replica.cc:330] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: stopping tablet replica
I20260807 18:10:51.665673 26762 ts_tablet_manager.cc:1905] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:51.665833 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26309
I20260807 18:10:51.666080 26761 raft_consensus.cc:2229] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:51.666724 26761 raft_consensus.cc:2258] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:51.670629 26761 ts_tablet_manager.cc:1905] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:51.671023 26760 ts_tablet_manager.cc:1905] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:51.674702 26762 ts_tablet_manager.cc:1918] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:51.674974 26762 log.cc:1196] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/8a6f52ec9de14e19bc56b551af8467bf
I20260807 18:10:51.675966 26762 ts_tablet_manager.cc:1939] T 8a6f52ec9de14e19bc56b551af8467bf P 463d55f653c747f0ae38f174c62a8524: Deleting consensus metadata
I20260807 18:10:51.678323 26230 catalog_manager.cc:4841] TS 463d55f653c747f0ae38f174c62a8524 (127.8.62.129:34283): tablet 8a6f52ec9de14e19bc56b551af8467bf (table test-table [id=779191d32325450fbb1a81c35e38b578]) successfully deleted
I20260807 18:10:51.679924 26761 ts_tablet_manager.cc:1918] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:51.680112 26761 log.cc:1196] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/8a6f52ec9de14e19bc56b551af8467bf
I20260807 18:10:51.680764 26761 ts_tablet_manager.cc:1939] T 8a6f52ec9de14e19bc56b551af8467bf P 968d79cec0b9428dbe9885c7dab4d50d: Deleting consensus metadata
I20260807 18:10:51.682896 26228 catalog_manager.cc:4841] TS 968d79cec0b9428dbe9885c7dab4d50d (127.8.62.131:44483): tablet 8a6f52ec9de14e19bc56b551af8467bf (table test-table [id=779191d32325450fbb1a81c35e38b578]) successfully deleted
I20260807 18:10:51.683766 26760 ts_tablet_manager.cc:1918] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:51.683997 26760 log.cc:1196] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/8a6f52ec9de14e19bc56b551af8467bf
I20260807 18:10:51.684613 26760 ts_tablet_manager.cc:1939] T 8a6f52ec9de14e19bc56b551af8467bf P 5650573bd9424722b0fdc35c165c09db: Deleting consensus metadata
I20260807 18:10:51.686710 26229 catalog_manager.cc:4841] TS 5650573bd9424722b0fdc35c165c09db (127.8.62.130:42991): tablet 8a6f52ec9de14e19bc56b551af8467bf (table test-table [id=779191d32325450fbb1a81c35e38b578]) successfully deleted
Aug 07 18:10:51 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:10:51.698426 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.667364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.8.62.129:34283)
0807 18:10:51.667571 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.667581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.673127 (+ 5546us) server_negotiation.cc:407] Connection header received
0807 18:10:51.673460 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.673467 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.673593 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.673678 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:51.679967 (+ 6289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.681418 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.684112 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.684860 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.685012 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.690496 (+ 5484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:51.690523 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:51.690547 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:51.690620 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:51.695010 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.695888 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.695901 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.695905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.696114 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.696535 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.696593 (+ 58us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.696598 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.697141 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:51.697469 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.697736 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.697987 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:10:51.699334 26436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:51.795930 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26309
I20260807 18:10:51.816488 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26448
Aug 07 18:10:51 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:10:51.836509 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.817468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.8.62.130:42991)
0807 18:10:51.817677 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.817684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.817709 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:10:51.817837 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.817844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.817957 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.818089 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:51.819444 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.821079 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.822544 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.823300 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.823456 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.828514 (+ 5058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:51.828539 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:51.828589 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:51.828654 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:51.833159 (+ 4505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.833983 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.833994 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.833998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.834199 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.834586 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.834592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.834596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.835128 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:51.835550 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.835789 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.836074 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 18:10:51.837430 26575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:51.952163 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26448
I20260807 18:10:51.973744 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26587
Aug 07 18:10:51 dist-test-slave-rnrw krb5kdc[26199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:10:51.993659 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:51.974744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40570 (local address 127.8.62.131:44483)
0807 18:10:51.976734 (+ 1990us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:51.976740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:51.976760 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:10:51.976822 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:51.976826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:51.976896 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:51.977015 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:51.978615 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.979740 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.981591 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:51.982046 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:51.982164 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:51.987413 (+ 5249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:51.987443 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:51.987460 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:51.987505 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:51.990518 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.991356 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.991364 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.991366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.991538 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:51.991996 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:51.992001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:51.992003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:51.992404 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:51.992783 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:51.993055 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 18:10:51.993258 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1847}
I20260807 18:10:51.994351 26712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:52.084928 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26587
I20260807 18:10:52.103644 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26212
I20260807 18:10:52.104892 26276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:52.219205 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26212
2026-08-07T18:10:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4594 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T18:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:10:52Z Disabled control of system clock
I20260807 18:10:52.307238 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37711
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:37711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:10:52.664043 26772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:52.664443 26772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:52.664592 26772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:52.672957 26772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:10:52.673101 26772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:52.673141 26772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:52.673204 26772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:10:52.673250 26772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:10:52.683097 26772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:37711
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:52.684432 26772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:52.686393 26772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:52.699071 26777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:52.699035 26778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:52.699137 26780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:52.700356 26772 server_base.cc:1034] running on GCE node
I20260807 18:10:52.701337 26772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:52.702958 26772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:52.704170 26772 hybrid_clock.cc:648] HybridClock initialized: now 1786126252704096 us; error 65 us; skew 500 ppm
I20260807 18:10:52.704627 26772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:10:52.707123 26772 webserver.cc:460] Webserver started at http://127.8.62.190:34345/ using document root <none> and password file <none>
I20260807 18:10:52.707937 26772 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:52.708062 26772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:52.708424 26772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:52.711578 26772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "03c37b1277d5497095486ff0224ce5e5"
format_stamp: "Formatted at 2026-08-07 18:10:52 on dist-test-slave-rnrw"
server_key: "ee1ff98a2e281eb838e4478977441930"
server_key_iv: "e684a219c164c905b3fc4df5f5c9f420"
server_key_version: "encryptionkey@0"
I20260807 18:10:52.712492 26772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "03c37b1277d5497095486ff0224ce5e5"
format_stamp: "Formatted at 2026-08-07 18:10:52 on dist-test-slave-rnrw"
server_key: "ee1ff98a2e281eb838e4478977441930"
server_key_iv: "e684a219c164c905b3fc4df5f5c9f420"
server_key_version: "encryptionkey@0"
I20260807 18:10:52.717937 26772 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 18:10:52.721813 26786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:52.723294 26772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 18:10:52.723497 26772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "03c37b1277d5497095486ff0224ce5e5"
format_stamp: "Formatted at 2026-08-07 18:10:52 on dist-test-slave-rnrw"
server_key: "ee1ff98a2e281eb838e4478977441930"
server_key_iv: "e684a219c164c905b3fc4df5f5c9f420"
server_key_version: "encryptionkey@0"
I20260807 18:10:52.723754 26772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:52.751467 26772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:52.752425 26772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:52.752821 26772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:52.773519 26772 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:37711
I20260807 18:10:52.773624 26839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:37711 every 8 connection(s)
I20260807 18:10:52.775096 26772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:10:52.776777 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26772
I20260807 18:10:52.777017 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:10:52.777776 8442 external_mini_cluster.cc:1428] Setting key c435d3a00402349212ce6da35d6e331a
I20260807 18:10:52.781401 26840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:52.798714 26840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: Bootstrap starting.
I20260807 18:10:52.801213 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:52.782159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.8.62.190:37711)
0807 18:10:52.787623 (+ 5464us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:52.787644 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:52.787810 (+ 166us) server_negotiation.cc:407] Connection header received
0807 18:10:52.789965 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:52.790004 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:52.790811 (+ 807us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:52.791355 (+ 544us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:52.791955 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:52.793985 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:52.795479 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:52.796705 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:52.796897 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:52.798112 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:52.798123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:52.798145 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:52.798937 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:52.799054 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:52.799087 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:52.799194 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:52.799436 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:10:52.799797 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4717,"thread_start_us":4408,"threads_started":1}
I20260807 18:10:52.803112 26840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:52.804292 26840 log.cc:824] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:52.807473 26840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: No bootstrap required, opened a new log
I20260807 18:10:52.815366 26840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } }
I20260807 18:10:52.816044 26840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:52.816151 26840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c37b1277d5497095486ff0224ce5e5, State: Initialized, Role: FOLLOWER
I20260807 18:10:52.816845 26840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } }
I20260807 18:10:52.817098 26840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:10:52.817219 26840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:10:52.817359 26840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:52.820092 26840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } }
I20260807 18:10:52.820693 26840 leader_election.cc:302] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03c37b1277d5497095486ff0224ce5e5; no voters:
I20260807 18:10:52.821300 26840 leader_election.cc:288] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:10:52.821455 26846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:52.822218 26846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [term 1 LEADER]: Becoming Leader. State: Replica: 03c37b1277d5497095486ff0224ce5e5, State: Running, Role: LEADER
I20260807 18:10:52.822819 26846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } }
I20260807 18:10:52.823770 26840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:10:52.825932 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c37b1277d5497095486ff0224ce5e5. Latest consensus state: current_term: 1 leader_uuid: "03c37b1277d5497095486ff0224ce5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } } }
I20260807 18:10:52.826498 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:52.829339 26847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03c37b1277d5497095486ff0224ce5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c37b1277d5497095486ff0224ce5e5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37711 } } }
I20260807 18:10:52.829792 26847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:52.832093 26853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:10:52.837905 26853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:10:52.848259 26853 catalog_manager.cc:1347] Generated new cluster ID: cf88a55b45194bd2a39143cb320f02d7
I20260807 18:10:52.848395 26853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:10:52.885089 26853 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:10:52.886732 26853 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:10:52.899365 26853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: Generated new TSK 0
I20260807 18:10:52.900457 26853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:10:52.916404 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--builtin_ntp_servers=127.8.62.148:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:10:53.279538 26865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:53.279841 26865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:53.279981 26865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:53.287714 26865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:53.287833 26865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:53.288003 26865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:10:53.298360 26865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:53.299875 26865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:53.301942 26865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:53.317126 26870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:53.317238 26871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:53.317106 26873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:53.319186 26865 server_base.cc:1034] running on GCE node
I20260807 18:10:53.319892 26865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:53.321029 26865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:53.322304 26865 hybrid_clock.cc:648] HybridClock initialized: now 1786126253322249 us; error 44 us; skew 500 ppm
I20260807 18:10:53.322698 26865 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:10:53.325321 26865 webserver.cc:460] Webserver started at http://127.8.62.129:34673/ using document root <none> and password file <none>
I20260807 18:10:53.326092 26865 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:53.326236 26865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:53.326593 26865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:53.330137 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "d52aec1345b34b4caab255f35a4510e5"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "f69b2625632c4431f2b257973de75aac"
server_key_iv: "c8c0faf343fb38dfa99b9b72ca686d1a"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.331087 26865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "d52aec1345b34b4caab255f35a4510e5"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "f69b2625632c4431f2b257973de75aac"
server_key_iv: "c8c0faf343fb38dfa99b9b72ca686d1a"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.336910 26865 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:10:53.341010 26879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.342640 26865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:10:53.342828 26865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "d52aec1345b34b4caab255f35a4510e5"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "f69b2625632c4431f2b257973de75aac"
server_key_iv: "c8c0faf343fb38dfa99b9b72ca686d1a"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.343056 26865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:53.366256 26865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:53.367231 26865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:53.367607 26865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:53.368775 26865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:53.370469 26865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:53.370604 26865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.370754 26865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:53.370857 26865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.407066 26865 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:34833
I20260807 18:10:53.407159 26992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:34833 every 8 connection(s)
I20260807 18:10:53.408488 26865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:10:53.418678 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26865
I20260807 18:10:53.418883 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:10:53.419337 8442 external_mini_cluster.cc:1428] Setting key dcb10c0f49066e1bd8987dbd17cd7086
I20260807 18:10:53.421698 26995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:53.411243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:57635 (local address 127.8.62.190:37711)
0807 18:10:53.411747 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:53.411754 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:53.412974 (+ 1220us) server_negotiation.cc:407] Connection header received
0807 18:10:53.414783 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:53.414787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:53.414859 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:53.414958 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:53.416027 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.416851 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.418547 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.419059 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.419196 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:53.420814 (+ 1618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:53.420819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:53.420822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:53.420841 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:53.420883 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:53.420895 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:53.420997 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:53.421242 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 18:10:53.421348 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":234,"threads_started":1}
I20260807 18:10:53.422341 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--builtin_ntp_servers=127.8.62.148:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:10:53.422904 26996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:53.411813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37711 (local address 127.8.62.129:57635)
0807 18:10:53.412819 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:53.412857 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:53.414528 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:53.415173 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:53.415181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:53.415275 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:53.415858 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:53.415875 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.417032 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.417036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:53.418380 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:53.418392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.419308 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.419315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:53.419615 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:53.420377 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:10:53.420380 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:53.420422 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:10:53.420510 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:53.421001 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:53.421016 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:53.421025 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 18:10:53.421230 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 18:10:53.421685 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":287,"threads_started":1}
I20260807 18:10:53.424733 26993 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37711
I20260807 18:10:53.425101 26993 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:53.425992 26993 heartbeater.cc:511] Master 127.8.62.190:37711 requested a full tablet report, sending...
I20260807 18:10:53.428582 26803 ts_manager.cc:194] Registered new tserver with Master: d52aec1345b34b4caab255f35a4510e5 (127.8.62.129:34833)
I20260807 18:10:53.430881 26803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.129:57635
W20260807 18:10:53.799664 26997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:53.800007 26997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:53.800235 26997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:53.809054 26997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:53.809178 26997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:53.809363 26997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:10:53.819052 26997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:53.820434 26997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:53.822381 26997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:53.836838 27002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:53.836915 27005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:53.836879 27003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:53.837738 26997 server_base.cc:1034] running on GCE node
I20260807 18:10:53.838395 26997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:53.839485 26997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:53.840732 26997 hybrid_clock.cc:648] HybridClock initialized: now 1786126253840663 us; error 55 us; skew 500 ppm
I20260807 18:10:53.841085 26997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:10:53.843570 26997 webserver.cc:460] Webserver started at http://127.8.62.130:37689/ using document root <none> and password file <none>
I20260807 18:10:53.844318 26997 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:53.844456 26997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:53.844828 26997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:53.847963 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "f35cf034cfb14f34a044e716a9f555c9"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "154b771fcec48388d8e4696b29956a03"
server_key_iv: "56b254bc68e55e843465aa0bd3a1046b"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.848893 26997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "f35cf034cfb14f34a044e716a9f555c9"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "154b771fcec48388d8e4696b29956a03"
server_key_iv: "56b254bc68e55e843465aa0bd3a1046b"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.854664 26997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:10:53.858671 27011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.860308 26997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:10:53.860620 26997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "f35cf034cfb14f34a044e716a9f555c9"
format_stamp: "Formatted at 2026-08-07 18:10:53 on dist-test-slave-rnrw"
server_key: "154b771fcec48388d8e4696b29956a03"
server_key_iv: "56b254bc68e55e843465aa0bd3a1046b"
server_key_version: "encryptionkey@0"
I20260807 18:10:53.860905 26997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:53.879026 26997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:53.880059 26997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:53.880414 26997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:53.881553 26997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:53.883284 26997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:53.883441 26997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.883589 26997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:53.883689 26997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:53.922250 26997 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:33483
I20260807 18:10:53.922379 27124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:33483 every 8 connection(s)
I20260807 18:10:53.923976 26997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:10:53.933497 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 26997
I20260807 18:10:53.933732 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:10:53.934394 8442 external_mini_cluster.cc:1428] Setting key 3f615d35e4eea9a2f2ce434103bf4029
I20260807 18:10:53.938048 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--builtin_ntp_servers=127.8.62.148:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:10:53.939742 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:53.927059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:39531 (local address 127.8.62.190:37711)
0807 18:10:53.927576 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:53.927583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:53.929057 (+ 1474us) server_negotiation.cc:407] Connection header received
0807 18:10:53.931122 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:53.931127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:53.931204 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:53.931313 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:53.932847 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.933700 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.936129 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.936784 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.936898 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:53.938870 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:53.938875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:53.938879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:53.938898 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:53.938943 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:53.938956 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:53.939036 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:53.939366 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 18:10:53.939461 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":242,"threads_started":1}
I20260807 18:10:53.941757 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:53.927748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37711 (local address 127.8.62.130:39531)
0807 18:10:53.928891 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:53.928934 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:53.930672 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:53.931683 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:53.931692 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:53.931821 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:53.932669 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:53.932692 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.933926 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.933932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:53.935967 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:53.935981 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:53.937054 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:53.937063 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:53.937418 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:53.938418 (+ 1000us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:10:53.938423 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:53.938490 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:10:53.938640 (+ 150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:53.939076 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:53.939090 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:53.939103 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 18:10:53.939353 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 18:10:53.939927 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":323,"threads_started":1}
I20260807 18:10:53.943673 27125 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37711
I20260807 18:10:53.944069 27125 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:53.945225 27125 heartbeater.cc:511] Master 127.8.62.190:37711 requested a full tablet report, sending...
I20260807 18:10:53.947018 26803 ts_manager.cc:194] Registered new tserver with Master: f35cf034cfb14f34a044e716a9f555c9 (127.8.62.130:33483)
I20260807 18:10:53.948042 26803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.130:39531
W20260807 18:10:54.312254 27129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:54.312659 27129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:54.312870 27129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:54.320529 27129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:54.320736 27129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:54.320917 27129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:10:54.330708 27129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37711
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:54.332669 27129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:54.334925 27129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:54.351045 27135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:54.351163 27134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:54.351166 27137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:54.352074 27129 server_base.cc:1034] running on GCE node
I20260807 18:10:54.352931 27129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:54.354102 27129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:54.355401 27129 hybrid_clock.cc:648] HybridClock initialized: now 1786126254355348 us; error 76 us; skew 500 ppm
I20260807 18:10:54.355806 27129 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:10:54.358431 27129 webserver.cc:460] Webserver started at http://127.8.62.131:37895/ using document root <none> and password file <none>
I20260807 18:10:54.359328 27129 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:54.359493 27129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:54.359875 27129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:54.363241 27129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "2ee6757b510b4fde88a49c663a923fee"
format_stamp: "Formatted at 2026-08-07 18:10:54 on dist-test-slave-rnrw"
server_key: "d91336a5fb9db9a5c56ac39683ca4241"
server_key_iv: "4765dedfd2176643f50a98aa9bf92861"
server_key_version: "encryptionkey@0"
I20260807 18:10:54.364168 27129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "2ee6757b510b4fde88a49c663a923fee"
format_stamp: "Formatted at 2026-08-07 18:10:54 on dist-test-slave-rnrw"
server_key: "d91336a5fb9db9a5c56ac39683ca4241"
server_key_iv: "4765dedfd2176643f50a98aa9bf92861"
server_key_version: "encryptionkey@0"
I20260807 18:10:54.370496 27129 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 18:10:54.374770 27143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:54.376407 27129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:10:54.376662 27129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "2ee6757b510b4fde88a49c663a923fee"
format_stamp: "Formatted at 2026-08-07 18:10:54 on dist-test-slave-rnrw"
server_key: "d91336a5fb9db9a5c56ac39683ca4241"
server_key_iv: "4765dedfd2176643f50a98aa9bf92861"
server_key_version: "encryptionkey@0"
I20260807 18:10:54.376931 27129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:54.394121 27129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:54.395040 27129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:54.395385 27129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:54.396459 27129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:54.397984 27129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:54.398130 27129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:54.398260 27129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:54.398350 27129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:54.433220 27129 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:36381
I20260807 18:10:54.433277 27257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:36381 every 8 connection(s)
I20260807 18:10:54.434701 26993 heartbeater.cc:503] Master 127.8.62.190:37711 was elected leader, sending a full tablet report...
I20260807 18:10:54.434857 27129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:10:54.444419 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27129
I20260807 18:10:54.444692 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:10:54.445171 8442 external_mini_cluster.cc:1428] Setting key f3391c8fd1b7938fef40e9bca9e0686b
I20260807 18:10:54.448953 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.437917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44071 (local address 127.8.62.190:37711)
0807 18:10:54.438136 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.438140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.439645 (+ 1505us) server_negotiation.cc:407] Connection header received
0807 18:10:54.441716 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.441722 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.441833 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.441969 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:54.443176 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.443947 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.445791 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.446323 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.446484 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.448099 (+ 1615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:54.448106 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:54.448117 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:54.448136 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:54.448168 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:54.448181 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:54.448280 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.448520 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.448647 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 18:10:54.450115 27260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.438460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37711 (local address 127.8.62.131:44071)
0807 18:10:54.439467 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.439507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.441386 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.442250 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.442259 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.442360 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:54.442993 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.443012 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.444117 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.444134 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.445624 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.445638 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.446571 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.446580 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.446914 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.447658 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:10:54.447660 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:54.447704 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:10:54.447790 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:54.448280 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:54.448295 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:54.448306 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.448512 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.448987 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":319,"threads_started":1}
I20260807 18:10:54.451529 27258 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37711
I20260807 18:10:54.451864 27258 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:54.452833 27258 heartbeater.cc:511] Master 127.8.62.190:37711 requested a full tablet report, sending...
I20260807 18:10:54.454313 26803 ts_manager.cc:194] Registered new tserver with Master: 2ee6757b510b4fde88a49c663a923fee (127.8.62.131:36381)
I20260807 18:10:54.455159 26803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.131:44071
I20260807 18:10:54.460917 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:10:54.475850 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.465971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51280 (local address 127.8.62.190:37711)
0807 18:10:54.466139 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.466143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.466536 (+ 393us) server_negotiation.cc:407] Connection header received
0807 18:10:54.466748 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.466751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.466819 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.466912 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:54.467456 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.468282 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.469781 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.470234 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.470336 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.475121 (+ 4785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:54.475127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:54.475130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:54.475153 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:54.475194 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:54.475208 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:54.475291 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.475423 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.475572 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 18:10:54.480206 26803 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:10:54.487712 26803 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:10:54.540740 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.501291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.8.62.129:34833)
0807 18:10:54.525229 (+ 23938us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.525243 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.528609 (+ 3366us) server_negotiation.cc:407] Connection header received
0807 18:10:54.529425 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.529433 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.529607 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.529809 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:54.531745 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.533384 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.537164 (+ 3780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.538776 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.538890 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.539200 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.540122 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.540310 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23546,"thread_start_us":23971,"threads_started":1}
I20260807 18:10:54.540740 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.503255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42428 (local address 127.8.62.130:33483)
0807 18:10:54.516754 (+ 13499us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.516768 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.529118 (+ 12350us) server_negotiation.cc:407] Connection header received
0807 18:10:54.529357 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.529365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.529607 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.529790 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:54.531234 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.533391 (+ 2157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.537167 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.538776 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.538890 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.539202 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.540122 (+ 920us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.540310 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13214,"thread_start_us":22850,"threads_started":1}
I20260807 18:10:54.541230 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.500725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:36381 (local address 127.0.0.1:43898)
0807 18:10:54.526459 (+ 25734us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.526539 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.526890 (+ 351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.530053 (+ 3163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.530064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.530105 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:54.530985 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.531043 (+ 58us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.534127 (+ 3084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.534135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.537408 (+ 3273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.537428 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.540275 (+ 2847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.540283 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.540455 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.540485 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.540611 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.540832 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25280,"mutex_wait_us":39,"thread_start_us":27996,"threads_started":1}
I20260807 18:10:54.541574 27271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.500966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55124)
0807 18:10:54.528424 (+ 27458us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.528467 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.529231 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.530111 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.530116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.530166 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:54.531522 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.531535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.533581 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.533586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.536890 (+ 3304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.536906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.539408 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.539415 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.539735 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.539891 (+ 156us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.540985 (+ 1094us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.541125 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27285,"mutex_wait_us":1083,"spinlock_wait_cycles":3968,"thread_start_us":27146,"threads_started":1}
I20260807 18:10:54.542013 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.502524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:33483 (local address 127.0.0.1:42428)
0807 18:10:54.528984 (+ 26460us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.529004 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.529159 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.530111 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.530116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.530156 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:54.530938 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.530958 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.533587 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.533590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.536890 (+ 3300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.536906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.539420 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.539425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.539731 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.539891 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.541449 (+ 1558us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.541615 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26331,"mutex_wait_us":481,"spinlock_wait_cycles":768,"thread_start_us":23216,"threads_started":1}
I20260807 18:10:54.542907 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.500992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.8.62.131:36381)
0807 18:10:54.528800 (+ 27808us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.528819 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.528854 (+ 35us) server_negotiation.cc:407] Connection header received
0807 18:10:54.528956 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.528964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.529431 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.529670 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:54.532076 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.533923 (+ 1847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.537677 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.539501 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.539678 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.540112 (+ 434us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.542163 (+ 2051us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.542413 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27635,"thread_start_us":27465,"threads_started":1}
I20260807 18:10:54.546015 27060 tablet_service.cc:1467] Processing CreateTablet for tablet 073ea27970ed41f289d7472c5947ca68 (DEFAULT_TABLE table=test-table [id=6f57ebb1a87f40b7a6e88d190fe7756c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:54.546010 26927 tablet_service.cc:1467] Processing CreateTablet for tablet 073ea27970ed41f289d7472c5947ca68 (DEFAULT_TABLE table=test-table [id=6f57ebb1a87f40b7a6e88d190fe7756c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:54.546947 27192 tablet_service.cc:1467] Processing CreateTablet for tablet 073ea27970ed41f289d7472c5947ca68 (DEFAULT_TABLE table=test-table [id=6f57ebb1a87f40b7a6e88d190fe7756c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:54.547731 26927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073ea27970ed41f289d7472c5947ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:54.548735 27192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073ea27970ed41f289d7472c5947ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:54.548807 27060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073ea27970ed41f289d7472c5947ca68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:54.570047 27276 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Bootstrap starting.
I20260807 18:10:54.574292 27278 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Bootstrap starting.
I20260807 18:10:54.577147 27276 tablet_bootstrap.cc:654] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:54.578193 27277 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Bootstrap starting.
I20260807 18:10:54.579114 27278 tablet_bootstrap.cc:654] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:54.580755 27278 log.cc:824] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:54.581341 27276 log.cc:824] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:54.582746 27277 tablet_bootstrap.cc:654] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:54.584378 27277 log.cc:824] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:54.592187 27276 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: No bootstrap required, opened a new log
I20260807 18:10:54.592244 27278 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: No bootstrap required, opened a new log
I20260807 18:10:54.592641 27276 ts_tablet_manager.cc:1397] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.010s
I20260807 18:10:54.592720 27278 ts_tablet_manager.cc:1397] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.005s
I20260807 18:10:54.593866 27277 tablet_bootstrap.cc:492] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: No bootstrap required, opened a new log
I20260807 18:10:54.594236 27277 ts_tablet_manager.cc:1397] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260807 18:10:54.605203 27277 raft_consensus.cc:348] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.605867 27277 raft_consensus.cc:374] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:54.606055 27277 raft_consensus.cc:729] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ee6757b510b4fde88a49c663a923fee, State: Initialized, Role: FOLLOWER
I20260807 18:10:54.604025 27276 raft_consensus.cc:348] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.606333 27276 raft_consensus.cc:374] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:54.606505 27276 raft_consensus.cc:729] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52aec1345b34b4caab255f35a4510e5, State: Initialized, Role: FOLLOWER
I20260807 18:10:54.606922 27277 consensus_queue.cc:260] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.606890 27278 raft_consensus.cc:348] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.607244 27276 consensus_queue.cc:260] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.607508 27278 raft_consensus.cc:374] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:54.607688 27278 raft_consensus.cc:729] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35cf034cfb14f34a044e716a9f555c9, State: Initialized, Role: FOLLOWER
I20260807 18:10:54.608505 27278 consensus_queue.cc:260] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.614389 27258 heartbeater.cc:503] Master 127.8.62.190:37711 was elected leader, sending a full tablet report...
I20260807 18:10:54.615545 27277 ts_tablet_manager.cc:1428] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Time spent starting tablet: real 0.021s user 0.014s sys 0.004s
I20260807 18:10:54.617434 27276 ts_tablet_manager.cc:1428] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Time spent starting tablet: real 0.025s user 0.010s sys 0.009s
I20260807 18:10:54.623306 27125 heartbeater.cc:503] Master 127.8.62.190:37711 was elected leader, sending a full tablet report...
I20260807 18:10:54.623744 27278 ts_tablet_manager.cc:1428] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Time spent starting tablet: real 0.031s user 0.009s sys 0.020s
W20260807 18:10:54.661422 26994 tablet.cc:2369] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:54.676666 27126 tablet.cc:2369] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:54.686774 27259 tablet.cc:2369] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:10:54.695312 27284 raft_consensus.cc:482] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:10:54.695735 27284 raft_consensus.cc:504] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.698439 27284 leader_election.cc:288] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ee6757b510b4fde88a49c663a923fee (127.8.62.131:36381), d52aec1345b34b4caab255f35a4510e5 (127.8.62.129:34833)
I20260807 18:10:54.706568 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.698969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45597 (local address 127.8.62.129:34833)
0807 18:10:54.699171 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.699178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.700157 (+ 979us) server_negotiation.cc:407] Connection header received
0807 18:10:54.700365 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.700370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.700441 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.700555 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:54.701257 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.702380 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.704499 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.705544 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.705740 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.705816 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.706143 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.706288 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:10:54.707664 27286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.698929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.8.62.130:45597)
0807 18:10:54.700026 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.700042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.700209 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.700589 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.700593 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.700618 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:54.701074 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.701085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.702569 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.702575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.704288 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.704301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.705754 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.705760 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.705943 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.705979 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.707075 (+ 1096us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.707241 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":68,"spinlock_wait_cycles":18816,"thread_start_us":787,"threads_started":1}
I20260807 18:10:54.708402 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.698908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:35427 (local address 127.8.62.131:36381)
0807 18:10:54.699125 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.699132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.699894 (+ 762us) server_negotiation.cc:407] Connection header received
0807 18:10:54.700261 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.700269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.700375 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.700443 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:54.701549 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.702591 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.705911 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.707140 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.707305 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.707376 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.707900 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.708089 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:10:54.708866 26948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073ea27970ed41f289d7472c5947ca68" candidate_uuid: "f35cf034cfb14f34a044e716a9f555c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52aec1345b34b4caab255f35a4510e5" is_pre_election: true
I20260807 18:10:54.709424 27285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.698855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:36381 (local address 127.8.62.130:35427)
0807 18:10:54.699747 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:54.699774 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:54.700047 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:54.700746 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:54.700753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:54.700804 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:54.701365 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.701380 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.702745 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.702753 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.705585 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:54.705607 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.707372 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.707381 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:54.707646 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.707705 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 18:10:54.708723 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0807 18:10:54.708933 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":3,"thread_start_us":483,"threads_started":1}
I20260807 18:10:54.709411 26948 raft_consensus.cc:2454] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35cf034cfb14f34a044e716a9f555c9 in term 0.
I20260807 18:10:54.710261 27013 leader_election.cc:302] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d52aec1345b34b4caab255f35a4510e5, f35cf034cfb14f34a044e716a9f555c9; no voters:
I20260807 18:10:54.710717 27284 raft_consensus.cc:2781] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:10:54.710886 27284 raft_consensus.cc:482] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:10:54.710723 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073ea27970ed41f289d7472c5947ca68" candidate_uuid: "f35cf034cfb14f34a044e716a9f555c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee6757b510b4fde88a49c663a923fee" is_pre_election: true
I20260807 18:10:54.711022 27284 raft_consensus.cc:3037] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:54.711341 27213 raft_consensus.cc:2454] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35cf034cfb14f34a044e716a9f555c9 in term 0.
I20260807 18:10:54.713629 27284 raft_consensus.cc:504] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.714370 27284 leader_election.cc:288] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [CANDIDATE]: Term 1 election: Requested vote from peers 2ee6757b510b4fde88a49c663a923fee (127.8.62.131:36381), d52aec1345b34b4caab255f35a4510e5 (127.8.62.129:34833)
I20260807 18:10:54.714877 26948 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073ea27970ed41f289d7472c5947ca68" candidate_uuid: "f35cf034cfb14f34a044e716a9f555c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52aec1345b34b4caab255f35a4510e5"
I20260807 18:10:54.714960 27213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073ea27970ed41f289d7472c5947ca68" candidate_uuid: "f35cf034cfb14f34a044e716a9f555c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ee6757b510b4fde88a49c663a923fee"
I20260807 18:10:54.715059 26948 raft_consensus.cc:3037] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:54.715221 27213 raft_consensus.cc:3037] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:54.717986 26948 raft_consensus.cc:2454] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35cf034cfb14f34a044e716a9f555c9 in term 1.
I20260807 18:10:54.718452 27013 leader_election.cc:302] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d52aec1345b34b4caab255f35a4510e5, f35cf034cfb14f34a044e716a9f555c9; no voters:
I20260807 18:10:54.718803 27284 raft_consensus.cc:2781] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:54.719229 27213 raft_consensus.cc:2454] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35cf034cfb14f34a044e716a9f555c9 in term 1.
I20260807 18:10:54.719411 27284 raft_consensus.cc:686] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 1 LEADER]: Becoming Leader. State: Replica: f35cf034cfb14f34a044e716a9f555c9, State: Running, Role: LEADER
I20260807 18:10:54.720036 27284 consensus_queue.cc:237] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } }
I20260807 18:10:54.725329 26802 catalog_manager.cc:5495] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f35cf034cfb14f34a044e716a9f555c9 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "f35cf034cfb14f34a044e716a9f555c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f35cf034cfb14f34a044e716a9f555c9" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 33483 } health_report { overall_health: HEALTHY } } }
I20260807 18:10:54.779800 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.771629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.8.62.130:33483)
0807 18:10:54.771898 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.771907 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.771932 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:10:54.772066 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.772075 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.772228 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.772429 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:10:54.773031 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.774319 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.776453 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.777169 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.777287 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.778059 (+ 772us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:10:54.778798 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:10:54.778931 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.779431 (+ 500us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.779516 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:10:54.792011 26948 raft_consensus.cc:1261] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 1 FOLLOWER]: Refusing update from remote peer f35cf034cfb14f34a044e716a9f555c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:54.792342 27213 raft_consensus.cc:1261] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 1 FOLLOWER]: Refusing update from remote peer f35cf034cfb14f34a044e716a9f555c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:54.793145 27284 consensus_queue.cc:1035] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52aec1345b34b4caab255f35a4510e5" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:54.793466 27287 consensus_queue.cc:1035] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ee6757b510b4fde88a49c663a923fee" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 36381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:54.810556 27290 mvcc.cc:205] Tried to move back new op lower bound from 7315973139617001472 to 7315973139338219520. Current Snapshot: MvccSnapshot[applied={T|T < 7315973139617001472}]
I20260807 18:10:54.842643 26802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:54.843029 26802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:54.848332 26802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03c37b1277d5497095486ff0224ce5e5: Sending DeleteTablet for 3 replicas of tablet 073ea27970ed41f289d7472c5947ca68
I20260807 18:10:54.849786 27192 tablet_service.cc:1514] Processing DeleteTablet for tablet 073ea27970ed41f289d7472c5947ca68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:54 UTC) from {username='slave'} at 127.0.0.1:43898
I20260807 18:10:54.849848 26927 tablet_service.cc:1514] Processing DeleteTablet for tablet 073ea27970ed41f289d7472c5947ca68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:54 UTC) from {username='slave'} at 127.0.0.1:55124
I20260807 18:10:54.850792 27300 tablet_replica.cc:330] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: stopping tablet replica
I20260807 18:10:54.851331 27300 raft_consensus.cc:2229] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:54.851958 27300 raft_consensus.cc:2258] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:54.852849 27060 tablet_service.cc:1514] Processing DeleteTablet for tablet 073ea27970ed41f289d7472c5947ca68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:54 UTC) from {username='slave'} at 127.0.0.1:42428
I20260807 18:10:54.854909 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26865
I20260807 18:10:54.861286 27302 tablet_replica.cc:330] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: stopping tablet replica
I20260807 18:10:54.861950 27302 raft_consensus.cc:2229] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:10:54.862874 27302 raft_consensus.cc:2258] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:54.866757 27302 ts_tablet_manager.cc:1905] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:54.871759 27300 ts_tablet_manager.cc:1905] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:54.879184 27300 ts_tablet_manager.cc:1918] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:54.879406 27300 log.cc:1196] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/073ea27970ed41f289d7472c5947ca68
I20260807 18:10:54.879979 27300 ts_tablet_manager.cc:1939] T 073ea27970ed41f289d7472c5947ca68 P 2ee6757b510b4fde88a49c663a923fee: Deleting consensus metadata
I20260807 18:10:54.881699 27302 ts_tablet_manager.cc:1918] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:54.882129 26789 catalog_manager.cc:4841] TS 2ee6757b510b4fde88a49c663a923fee (127.8.62.131:36381): tablet 073ea27970ed41f289d7472c5947ca68 (table test-table [id=6f57ebb1a87f40b7a6e88d190fe7756c]) successfully deleted
I20260807 18:10:54.882417 27301 tablet_replica.cc:330] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: stopping tablet replica
I20260807 18:10:54.882684 27302 log.cc:1196] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/073ea27970ed41f289d7472c5947ca68
I20260807 18:10:54.882973 27301 raft_consensus.cc:2229] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:54.883576 27301 raft_consensus.cc:2258] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:54.884902 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:54.856693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.8.62.129:34833)
0807 18:10:54.856921 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:54.856929 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:54.870787 (+ 13858us) server_negotiation.cc:407] Connection header received
0807 18:10:54.870999 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:54.871007 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:54.871143 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:54.871259 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:54.872096 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.873368 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.877034 (+ 3666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:54.881166 (+ 4132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:54.881342 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:54.882938 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:54.882952 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:54.882977 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:54.883806 (+ 829us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:54.883869 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:54.883897 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:54.884005 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:54.884338 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 18:10:54.884467 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:10:54.885108 27302 ts_tablet_manager.cc:1939] T 073ea27970ed41f289d7472c5947ca68 P f35cf034cfb14f34a044e716a9f555c9: Deleting consensus metadata
I20260807 18:10:54.886133 27301 ts_tablet_manager.cc:1905] T 073ea27970ed41f289d7472c5947ca68 P d52aec1345b34b4caab255f35a4510e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:54.886494 26988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:54.887696 26790 catalog_manager.cc:4841] TS f35cf034cfb14f34a044e716a9f555c9 (127.8.62.130:33483): tablet 073ea27970ed41f289d7472c5947ca68 (table test-table [id=6f57ebb1a87f40b7a6e88d190fe7756c]) successfully deleted
I20260807 18:10:54.982339 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26865
I20260807 18:10:55.002560 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26997
W20260807 18:10:55.002622 26788 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.62.129:34833 (error 108)
I20260807 18:10:55.004598 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.004077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55144)
0807 18:10:55.004227 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.004369 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:10:55.004900 26788 catalog_manager.cc:4584] TS d52aec1345b34b4caab255f35a4510e5 (127.8.62.129:34833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 073ea27970ed41f289d7472c5947ca68: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
I20260807 18:10:55.011771 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.003888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42446 (local address 127.8.62.130:33483)
0807 18:10:55.004079 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:55.004084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:55.004382 (+ 298us) server_negotiation.cc:407] Connection header received
0807 18:10:55.004621 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:55.004630 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:55.004755 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:55.004886 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:55.005392 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:55.006304 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:55.008203 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:55.008838 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:55.008959 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:55.010250 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:55.010259 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:55.010274 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:55.010923 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:55.010974 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:55.010996 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:55.011087 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:55.011250 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 18:10:55.011339 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:10:55.012643 27120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:55.058954 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.058348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55156)
0807 18:10:55.058650 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.058766 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260807 18:10:55.060314 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.059858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55158)
0807 18:10:55.060069 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.060171 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 18:10:55.121263 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.120586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55164)
0807 18:10:55.120863 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.120993 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:10:55.122627 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.122157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55180)
0807 18:10:55.122365 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.122471 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:10:55.132414 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26997
I20260807 18:10:55.154042 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27129
I20260807 18:10:55.161793 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.154858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43902 (local address 127.8.62.131:36381)
0807 18:10:55.155044 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:55.155049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:55.155074 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:10:55.155190 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:55.155194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:55.155287 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:55.155383 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:55.155822 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:55.156685 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:55.158313 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:55.158767 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:55.158881 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:55.160020 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:55.160030 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:55.160046 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:10:55.160868 (+ 822us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:55.160922 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:10:55.160944 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:55.161059 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:55.161278 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 18:10:55.161392 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:10:55.162648 27253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:55.204032 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.203376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55184)
0807 18:10:55.203656 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.203775 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 18:10:55.205299 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.204828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55190)
0807 18:10:55.205047 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.205149 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 18:10:55.255776 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27129
I20260807 18:10:55.274147 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 26772
I20260807 18:10:55.275331 26835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:55.387874 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 26772
I20260807 18:10:55.387948 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:55.387136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34833 (local address 127.0.0.1:55200)
0807 18:10:55.387514 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:55.387702 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
2026-08-07T18:10:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3165 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:10:55 dist-test-slave-rnrw krb5kdc[27311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:10:55 dist-test-slave-rnrw krb5kdc[27311](info): set up 2 sockets
Aug 07 18:10:55 dist-test-slave-rnrw krb5kdc[27311](info): commencing operation
krb5kdc: starting...
W20260807 18:10:56.631305 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:10:56 dist-test-slave-rnrw krb5kdc[27311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:10:56.925015 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40343
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:40343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:57.285784 27326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:57.286121 27326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:57.286300 27326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:57.293963 27326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:10:57.294095 27326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:57.294186 27326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:57.294275 27326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:10:57.294368 27326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:10:57.304694 27326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:40343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:40343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:57.306097 27326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:57.307492 27326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:57.320904 27334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:57.320935 27332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:57.320957 27331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:57.321897 27326 server_base.cc:1034] running on GCE node
I20260807 18:10:57.322903 27326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:57.324424 27326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:57.325686 27326 hybrid_clock.cc:648] HybridClock initialized: now 1786126257325633 us; error 50 us; skew 500 ppm
I20260807 18:10:57.326086 27326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:57 dist-test-slave-rnrw krb5kdc[27311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126257, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:57.329455 27326 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:10:57.331106 27326 webserver.cc:460] Webserver started at http://127.8.62.190:41013/ using document root <none> and password file <none>
I20260807 18:10:57.331873 27326 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:57.331993 27326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:57.332330 27326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:57.335632 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "1708f72c64dc9f65da551b9daee7db4e"
server_key_iv: "96bc3d07dd7dafac0a76aea789229bc1"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.336448 27326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "1708f72c64dc9f65da551b9daee7db4e"
server_key_iv: "96bc3d07dd7dafac0a76aea789229bc1"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.342379 27326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:10:57.346308 27341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:57.347790 27326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:10:57.347988 27326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "1708f72c64dc9f65da551b9daee7db4e"
server_key_iv: "96bc3d07dd7dafac0a76aea789229bc1"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.348197 27326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:57.363843 27326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:57.368266 27326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:57.368724 27326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:57.389199 27326 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:40343
I20260807 18:10:57.389294 27393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:40343 every 8 connection(s)
I20260807 18:10:57.390733 27326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:10:57.393442 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27326
I20260807 18:10:57.393662 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:10:57.394255 8442 external_mini_cluster.cc:1428] Setting key 3d22dd064ef6b54ff07f31b784cdf164
I20260807 18:10:57.395457 27394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:10:57 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:57.410655 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: Bootstrap starting.
I20260807 18:10:57.414388 27394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:57.415513 27394 log.cc:824] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:57.419380 27394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: No bootstrap required, opened a new log
I20260807 18:10:57.420055 27397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:57.396221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36016 (local address 127.8.62.190:40343)
0807 18:10:57.397125 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:57.397137 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:57.397179 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:10:57.398787 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:57.398810 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:57.399231 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:57.399579 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:57.401546 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:57.402925 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:57.404934 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:57.405642 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:57.405800 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:57.411024 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:57.411034 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:57.411051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:57.411093 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:57.414999 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:57.415927 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:57.415936 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:57.415943 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:57.416172 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:57.416717 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:57.416728 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:57.416731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:57.417418 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:57.417684 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:57.418292 (+ 608us) server_negotiation.cc:299] Negotiation successful
0807 18:10:57.418865 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":471,"threads_started":1}
I20260807 18:10:57.429366 27394 raft_consensus.cc:348] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } }
I20260807 18:10:57.429980 27394 raft_consensus.cc:374] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:57.430150 27394 raft_consensus.cc:729] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54f5ced4cbf4fa295e1de8238fbc3b4, State: Initialized, Role: FOLLOWER
I20260807 18:10:57.430891 27394 consensus_queue.cc:260] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } }
I20260807 18:10:57.431136 27394 raft_consensus.cc:388] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:10:57.431243 27394 raft_consensus.cc:482] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:10:57.431387 27394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:57.434001 27394 raft_consensus.cc:504] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } }
I20260807 18:10:57.434545 27394 leader_election.cc:302] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e54f5ced4cbf4fa295e1de8238fbc3b4; no voters:
I20260807 18:10:57.435199 27394 leader_election.cc:288] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:10:57.435359 27399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:57.436308 27399 raft_consensus.cc:686] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [term 1 LEADER]: Becoming Leader. State: Replica: e54f5ced4cbf4fa295e1de8238fbc3b4, State: Running, Role: LEADER
I20260807 18:10:57.437022 27399 consensus_queue.cc:237] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } }
I20260807 18:10:57.437937 27394 sys_catalog.cc:564] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:10:57.440481 27400 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } } }
I20260807 18:10:57.441015 27400 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:57.440507 27401 sys_catalog.cc:455] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e54f5ced4cbf4fa295e1de8238fbc3b4. Latest consensus state: current_term: 1 leader_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e54f5ced4cbf4fa295e1de8238fbc3b4" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 40343 } } }
I20260807 18:10:57.441394 27401 sys_catalog.cc:458] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4 [sys.catalog]: This master's current role is: LEADER
I20260807 18:10:57.442191 27406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:10:57.449291 27406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:10:57.459816 27406 catalog_manager.cc:1347] Generated new cluster ID: 8b17af1de4d64985a3539b9e50a2e8ed
I20260807 18:10:57.459964 27406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:10:57.470603 27406 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:10:57.472992 27406 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:10:57.482218 27406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: Generated new TSK 0
I20260807 18:10:57.483361 27406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:10:57.548321 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:57.896613 27420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:57.896952 27420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:57.897096 27420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:57.904739 27420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:57.904862 27420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:57.905046 27420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:10:57.915426 27420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:57.916826 27420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:57.918233 27420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:57.931697 27425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:57.931780 27428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:57.931738 27426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:57.932765 27420 server_base.cc:1034] running on GCE node
I20260807 18:10:57.933470 27420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:57.934438 27420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:57.935683 27420 hybrid_clock.cc:648] HybridClock initialized: now 1786126257935585 us; error 55 us; skew 500 ppm
I20260807 18:10:57.936057 27420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:57 dist-test-slave-rnrw krb5kdc[27311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126257, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:57.939532 27420 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:10:57.941120 27420 webserver.cc:460] Webserver started at http://127.8.62.129:42973/ using document root <none> and password file <none>
I20260807 18:10:57.941821 27420 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:57.941931 27420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:57.942196 27420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:57.945600 27420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "63e824c4033248518a357b5e37812561"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "d3b9be063abee602f687d17e7c5724de"
server_key_iv: "87e43b1ad593016f11330bdf03d4783c"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.946436 27420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "63e824c4033248518a357b5e37812561"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "d3b9be063abee602f687d17e7c5724de"
server_key_iv: "87e43b1ad593016f11330bdf03d4783c"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.951957 27420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:10:57.955737 27435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:57.957232 27420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:10:57.957419 27420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "63e824c4033248518a357b5e37812561"
format_stamp: "Formatted at 2026-08-07 18:10:57 on dist-test-slave-rnrw"
server_key: "d3b9be063abee602f687d17e7c5724de"
server_key_iv: "87e43b1ad593016f11330bdf03d4783c"
server_key_version: "encryptionkey@0"
I20260807 18:10:57.957614 27420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:57.972796 27420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:57.977082 27420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:57.977394 27420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:57.978434 27420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:57.979844 27420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:57.979919 27420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:57.980100 27420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:57.980185 27420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:58.014765 27420 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:40867
I20260807 18:10:58.014858 27549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:40867 every 8 connection(s)
I20260807 18:10:58.016114 27420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:10:58.017557 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27420
I20260807 18:10:58.017762 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:10:58.018365 8442 external_mini_cluster.cc:1428] Setting key f993942c1094cc28dcadfb54567d0ef4
Aug 07 18:10:58 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126257, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:58.038597 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:58.019110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:33349 (local address 127.8.62.190:40343)
0807 18:10:58.019622 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:58.019629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:58.020477 (+ 848us) server_negotiation.cc:407] Connection header received
0807 18:10:58.022570 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:58.022575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:58.022666 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:58.022807 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:58.024445 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.025264 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.027020 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.027459 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.027615 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:58.032214 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:58.032221 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:58.032225 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:58.032268 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:58.035090 (+ 2822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:58.035949 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:58.035955 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:58.035961 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:58.036107 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:58.036651 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:58.036655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:58.036657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:58.036941 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:58.037121 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:58.037913 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 18:10:58.038175 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":251,"threads_started":1}
I20260807 18:10:58.039858 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:58.019538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40343 (local address 127.8.62.129:33349)
0807 18:10:58.020315 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:58.020355 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:58.022278 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:58.023062 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:58.023072 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:58.023634 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:58.024251 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:58.024273 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.025465 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.025469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:58.026838 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:58.026852 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.027728 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.027745 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:58.028008 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:58.028817 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:58.028842 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:58.031896 (+ 3054us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:58.035278 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:58.035289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:58.035316 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:58.035807 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:58.036234 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:58.036239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:58.036243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:58.036464 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:58.037087 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:58.037096 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:58.037498 (+ 402us) client_negotiation.cc:769] Sending connection context
0807 18:10:58.037801 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 18:10:58.038491 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":335,"threads_started":1}
I20260807 18:10:58.041760 27550 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40343
I20260807 18:10:58.042104 27550 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:58.042866 27550 heartbeater.cc:511] Master 127.8.62.190:40343 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
I20260807 18:10:58.045472 27358 ts_manager.cc:194] Registered new tserver with Master: 63e824c4033248518a357b5e37812561 (127.8.62.129:40867)
I20260807 18:10:58.047114 27358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:33349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:10:58.084405 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:58.430661 27556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:58.430950 27556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:58.431116 27556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:58.438648 27556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:58.438779 27556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:58.438999 27556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:10:58.449313 27556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:58.450718 27556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:58.452174 27556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:58.466069 27564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:58.466146 27562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:58.466102 27561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:58.466892 27556 server_base.cc:1034] running on GCE node
I20260807 18:10:58.467517 27556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:58.468461 27556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:58.469722 27556 hybrid_clock.cc:648] HybridClock initialized: now 1786126258469675 us; error 43 us; skew 500 ppm
I20260807 18:10:58.470073 27556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:58 dist-test-slave-rnrw krb5kdc[27311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:58.473367 27556 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:10:58.474936 27556 webserver.cc:460] Webserver started at http://127.8.62.130:37207/ using document root <none> and password file <none>
I20260807 18:10:58.475690 27556 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:58.475840 27556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:58.476166 27556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:58.479292 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "ac3fb93d6af14eceba3464831152cc83"
format_stamp: "Formatted at 2026-08-07 18:10:58 on dist-test-slave-rnrw"
server_key: "53eabc7b940f1a7a43cd76cc28cd2bc3"
server_key_iv: "af671e0062aa9bfdf92f28dbf4aa908b"
server_key_version: "encryptionkey@0"
I20260807 18:10:58.480140 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "ac3fb93d6af14eceba3464831152cc83"
format_stamp: "Formatted at 2026-08-07 18:10:58 on dist-test-slave-rnrw"
server_key: "53eabc7b940f1a7a43cd76cc28cd2bc3"
server_key_iv: "af671e0062aa9bfdf92f28dbf4aa908b"
server_key_version: "encryptionkey@0"
I20260807 18:10:58.485780 27556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 18:10:58.489471 27571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:58.490880 27556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:10:58.491079 27556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "ac3fb93d6af14eceba3464831152cc83"
format_stamp: "Formatted at 2026-08-07 18:10:58 on dist-test-slave-rnrw"
server_key: "53eabc7b940f1a7a43cd76cc28cd2bc3"
server_key_iv: "af671e0062aa9bfdf92f28dbf4aa908b"
server_key_version: "encryptionkey@0"
I20260807 18:10:58.491289 27556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:58.518270 27556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:58.522979 27556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:58.523313 27556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:58.524407 27556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:58.525970 27556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:58.526089 27556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:58.526254 27556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:58.526335 27556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:58.563408 27556 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:35423
I20260807 18:10:58.563475 27686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:35423 every 8 connection(s)
I20260807 18:10:58.564939 27556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:10:58.572978 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27556
I20260807 18:10:58.573200 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:10:58.573645 8442 external_mini_cluster.cc:1428] Setting key 79c09651be25305069e75ce602e701e9
Aug 07 18:10:58 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126258, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:58.588717 27689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:58.567780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:56453 (local address 127.8.62.190:40343)
0807 18:10:58.568267 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:58.568272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:58.569362 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 18:10:58.571503 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:58.571510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:58.571617 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:58.571757 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:58.573690 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.574799 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.576943 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.577441 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.577606 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:58.581870 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:58.581876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:58.581880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:58.581920 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:58.584921 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:58.585894 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:58.585899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:58.585902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:58.586053 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:58.586594 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:58.586599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:58.586601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:58.586932 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:58.587117 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:58.587950 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 18:10:58.588238 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":253,"threads_started":1}
I20260807 18:10:58.589849 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:58.568357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40343 (local address 127.8.62.130:56453)
0807 18:10:58.569176 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:58.569229 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:58.571161 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:58.572047 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:58.572058 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:58.572733 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:58.573490 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:58.573512 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.575025 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.575030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:58.576758 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:58.576774 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:58.577737 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:58.577745 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:58.578022 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:58.578813 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:58.578839 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:58.581564 (+ 2725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:58.585111 (+ 3547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:58.585122 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:58.585143 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:58.585719 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:58.586191 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:58.586197 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:58.586201 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:58.586422 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:58.587096 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:58.587105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:58.587525 (+ 420us) client_negotiation.cc:769] Sending connection context
0807 18:10:58.587838 (+ 313us) client_negotiation.cc:240] Negotiation successful
0807 18:10:58.588489 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":348,"threads_started":1}
I20260807 18:10:58.591625 27687 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40343
I20260807 18:10:58.591998 27687 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:58.592891 27687 heartbeater.cc:511] Master 127.8.62.190:40343 requested a full tablet report, sending...
I20260807 18:10:58.594671 27358 ts_manager.cc:194] Registered new tserver with Master: ac3fb93d6af14eceba3464831152cc83 (127.8.62.130:35423)
I20260807 18:10:58.596048 27358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:56453
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:10:58.635924 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:41907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:10:58.985877 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:10:58.986172 27693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:10:58.986269 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:10:58.993821 27693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:10:58.993945 27693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:10:58.994064 27693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:10:59.004024 27693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:41907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:40343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:10:59.005411 27693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:10:59.006798 27693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:10:59.020558 27698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:59.020439 27699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:10:59.021183 27701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:10:59.021638 27693 server_base.cc:1034] running on GCE node
I20260807 18:10:59.022262 27693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:10:59.023343 27693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:10:59.024667 27693 hybrid_clock.cc:648] HybridClock initialized: now 1786126259024589 us; error 68 us; skew 500 ppm
I20260807 18:10:59.025022 27693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:10:59 dist-test-slave-rnrw krb5kdc[27311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:10:59.028333 27693 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:10:59.030084 27693 webserver.cc:460] Webserver started at http://127.8.62.131:34983/ using document root <none> and password file <none>
I20260807 18:10:59.030823 27693 fs_manager.cc:359] Metadata directory not provided
I20260807 18:10:59.030977 27693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:10:59.031311 27693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:10:59.034431 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "97ff6f0c21754d80b2df64ac1a7da13a"
format_stamp: "Formatted at 2026-08-07 18:10:59 on dist-test-slave-rnrw"
server_key: "daabd7721662e5deebbf6d27c278ac27"
server_key_iv: "63ec636db78389002880761c1e3588c1"
server_key_version: "encryptionkey@0"
I20260807 18:10:59.035270 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "97ff6f0c21754d80b2df64ac1a7da13a"
format_stamp: "Formatted at 2026-08-07 18:10:59 on dist-test-slave-rnrw"
server_key: "daabd7721662e5deebbf6d27c278ac27"
server_key_iv: "63ec636db78389002880761c1e3588c1"
server_key_version: "encryptionkey@0"
I20260807 18:10:59.041054 27693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 18:10:59.044924 27709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:59.046347 27693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:10:59.046547 27693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "97ff6f0c21754d80b2df64ac1a7da13a"
format_stamp: "Formatted at 2026-08-07 18:10:59 on dist-test-slave-rnrw"
server_key: "daabd7721662e5deebbf6d27c278ac27"
server_key_iv: "63ec636db78389002880761c1e3588c1"
server_key_version: "encryptionkey@0"
I20260807 18:10:59.046764 27693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:10:59.051759 27550 heartbeater.cc:503] Master 127.8.62.190:40343 was elected leader, sending a full tablet report...
I20260807 18:10:59.070132 27693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:10:59.075107 27693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:10:59.075456 27693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:10:59.076584 27693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:10:59.078061 27693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:10:59.078182 27693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:59.078357 27693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:10:59.078440 27693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:10:59.116819 27693 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:34285
I20260807 18:10:59.116880 27824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:34285 every 8 connection(s)
I20260807 18:10:59.118309 27693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:10:59.124826 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27693
I20260807 18:10:59.125087 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:10:59.125617 8442 external_mini_cluster.cc:1428] Setting key f081fd583c48cff4c195470de852860d
Aug 07 18:10:59 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126259, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:10:59.141724 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.121253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:60887 (local address 127.8.62.190:40343)
0807 18:10:59.121721 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.121726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.123017 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 18:10:59.125149 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.125156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.125278 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.125421 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:59.127391 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.128273 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.130177 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.130596 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.130746 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.134723 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:59.134730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:59.134733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:59.134770 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:59.137794 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.138878 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.138887 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.138894 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.139069 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.139540 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.139546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.139549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.139941 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:59.140164 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.140999 (+ 835us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.141270 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":228,"threads_started":1}
I20260807 18:10:59.142503 27828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.121907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:40343 (local address 127.8.62.131:60887)
0807 18:10:59.122808 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.122868 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.124799 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.125717 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.125726 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.126465 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:10:59.127192 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.127228 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.128451 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.128456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.130006 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.130020 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.130852 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.130860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.131109 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.131835 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:10:59.131859 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:10:59.134455 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:10:59.137977 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:59.137986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:59.138001 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:59.138660 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:59.139184 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:10:59.139188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:10:59.139190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:10:59.139395 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:10:59.140124 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:10:59.140130 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:10:59.140491 (+ 361us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.140825 (+ 334us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.141273 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":409,"threads_started":1}
I20260807 18:10:59.144198 27825 heartbeater.cc:347] Connected to a master server at 127.8.62.190:40343
I20260807 18:10:59.144529 27825 heartbeater.cc:464] Registering TS with master...
I20260807 18:10:59.145376 27825 heartbeater.cc:511] Master 127.8.62.190:40343 requested a full tablet report, sending...
I20260807 18:10:59.147053 27358 ts_manager.cc:194] Registered new tserver with Master: 97ff6f0c21754d80b2df64ac1a7da13a (127.8.62.131:34285)
I20260807 18:10:59.147923 27358 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:60887
I20260807 18:10:59.154830 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:10:59.174695 27827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.159296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39320 (local address 127.8.62.190:40343)
0807 18:10:59.159517 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.159522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.159919 (+ 397us) server_negotiation.cc:407] Connection header received
0807 18:10:59.160064 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.160069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.160144 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.160247 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:59.161990 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.162931 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.164490 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.165036 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.165163 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.168045 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:59.168053 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:59.168057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:59.168100 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:59.171304 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.172366 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.172371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.172374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.172618 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.173140 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.173144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.173146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.173508 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:59.173762 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.174210 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.174381 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:10:59.179400 27358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:10:59.187028 27358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:10:59.227000 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.201808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:40867 (local address 127.0.0.1:34184)
0807 18:10:59.216014 (+ 14206us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.216036 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.216181 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.217225 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.217234 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.217278 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:59.217845 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.217864 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.220865 (+ 3001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.220874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.223796 (+ 2922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.223812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.225884 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.225891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.226105 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.226174 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.226326 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.226550 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14046,"thread_start_us":13944,"threads_started":1}
I20260807 18:10:59.227912 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.202255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.8.62.129:40867)
0807 18:10:59.215724 (+ 13469us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.215735 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.216318 (+ 583us) server_negotiation.cc:407] Connection header received
0807 18:10:59.216627 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.216636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.216930 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.218134 (+ 1204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:59.218471 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.220651 (+ 2180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.224052 (+ 3401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.225666 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.225829 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.226109 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.227326 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.227548 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13052,"thread_start_us":12852,"threads_started":1}
I20260807 18:10:59.232523 27485 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca84fd77251414785705ea979b1949c (DEFAULT_TABLE table=test-table [id=272d924b51844b1d83b4c1c32a26d1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:59.234297 27485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca84fd77251414785705ea979b1949c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:59.234349 27842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.217166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:34285 (local address 127.0.0.1:32988)
0807 18:10:59.225118 (+ 7952us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.225143 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.225287 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.226598 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.226610 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.226635 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:59.227076 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.227095 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.229152 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.229157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.231275 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.231291 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.233452 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.233461 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.233640 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.233667 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.233741 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.233908 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7789,"mutex_wait_us":1,"thread_start_us":7350,"threads_started":1}
I20260807 18:10:59.235214 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.217176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32988 (local address 127.8.62.131:34285)
0807 18:10:59.225348 (+ 8172us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.225360 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.225414 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:10:59.225555 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.225570 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.225957 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.226262 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:59.227351 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.228989 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.231447 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.233075 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.233221 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.234534 (+ 1313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.234624 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.234806 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7965,"thread_start_us":7790,"threads_started":1}
I20260807 18:10:59.235530 27839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.201670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35423 (local address 127.0.0.1:47394)
0807 18:10:59.214283 (+ 12613us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.214342 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.214605 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.227560 (+ 12955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.227564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.227591 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:59.227973 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.227987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.230166 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.230172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.232195 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.232212 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.234768 (+ 2556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.234774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.234938 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.234988 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.235083 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.235210 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12248,"thread_start_us":14048,"threads_started":1}
I20260807 18:10:59.236274 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.201960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.8.62.130:35423)
0807 18:10:59.226561 (+ 24601us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.226571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.226601 (+ 30us) server_negotiation.cc:407] Connection header received
0807 18:10:59.226699 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.226710 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.226959 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.227305 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:59.228312 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.230009 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.232384 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.234278 (+ 1894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.234428 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.234623 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.235815 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.235999 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24429,"thread_start_us":24327,"threads_started":1}
I20260807 18:10:59.239444 27757 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca84fd77251414785705ea979b1949c (DEFAULT_TABLE table=test-table [id=272d924b51844b1d83b4c1c32a26d1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:59.240113 27620 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca84fd77251414785705ea979b1949c (DEFAULT_TABLE table=test-table [id=272d924b51844b1d83b4c1c32a26d1ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:10:59.240991 27757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca84fd77251414785705ea979b1949c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:59.241704 27620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca84fd77251414785705ea979b1949c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:10:59.250604 27844 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Bootstrap starting.
I20260807 18:10:59.256258 27844 tablet_bootstrap.cc:654] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:59.257682 27846 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Bootstrap starting.
I20260807 18:10:59.258081 27844 log.cc:824] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:59.259792 27845 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Bootstrap starting.
I20260807 18:10:59.262202 27844 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: No bootstrap required, opened a new log
I20260807 18:10:59.262352 27846 tablet_bootstrap.cc:654] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:59.262519 27844 ts_tablet_manager.cc:1397] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260807 18:10:59.263199 27845 tablet_bootstrap.cc:654] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Neither blocks nor log segments found. Creating new log.
I20260807 18:10:59.264587 27846 log.cc:824] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:59.264917 27845 log.cc:824] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Log is configured to *not* fsync() on all Append() calls
I20260807 18:10:59.268328 27846 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: No bootstrap required, opened a new log
I20260807 18:10:59.268693 27846 ts_tablet_manager.cc:1397] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260807 18:10:59.271224 27845 tablet_bootstrap.cc:492] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: No bootstrap required, opened a new log
I20260807 18:10:59.271549 27845 ts_tablet_manager.cc:1397] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260807 18:10:59.273164 27844 raft_consensus.cc:348] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.273656 27844 raft_consensus.cc:374] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:59.273808 27844 raft_consensus.cc:729] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e824c4033248518a357b5e37812561, State: Initialized, Role: FOLLOWER
I20260807 18:10:59.274509 27844 consensus_queue.cc:260] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.279495 27844 ts_tablet_manager.cc:1428] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Time spent starting tablet: real 0.017s user 0.014s sys 0.002s
I20260807 18:10:59.280346 27846 raft_consensus.cc:348] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.280838 27846 raft_consensus.cc:374] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:59.280961 27846 raft_consensus.cc:729] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97ff6f0c21754d80b2df64ac1a7da13a, State: Initialized, Role: FOLLOWER
I20260807 18:10:59.281687 27846 consensus_queue.cc:260] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.282896 27825 heartbeater.cc:503] Master 127.8.62.190:40343 was elected leader, sending a full tablet report...
I20260807 18:10:59.282860 27845 raft_consensus.cc:348] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.283399 27845 raft_consensus.cc:374] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:10:59.283541 27845 raft_consensus.cc:729] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3fb93d6af14eceba3464831152cc83, State: Initialized, Role: FOLLOWER
I20260807 18:10:59.283661 27846 ts_tablet_manager.cc:1428] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260807 18:10:59.284286 27845 consensus_queue.cc:260] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.285539 27687 heartbeater.cc:503] Master 127.8.62.190:40343 was elected leader, sending a full tablet report...
I20260807 18:10:59.286553 27845 ts_tablet_manager.cc:1428] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
W20260807 18:10:59.317262 27688 tablet.cc:2369] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:59.370249 27826 tablet.cc:2369] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:10:59.519248 27551 tablet.cc:2369] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:10:59.523500 27851 raft_consensus.cc:482] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:10:59.523806 27851 raft_consensus.cc:504] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.525734 27851 leader_election.cc:288] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac3fb93d6af14eceba3464831152cc83 (127.8.62.130:35423), 63e824c4033248518a357b5e37812561 (127.8.62.129:40867)
I20260807 18:10:59.532830 27828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.525999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35423 (local address 127.8.62.131:32851)
0807 18:10:59.526342 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.526363 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.526537 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.527030 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.527037 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.527071 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:59.527302 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.527315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.528890 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.528897 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.530644 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.530667 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.531947 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.531954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.532166 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.532202 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.532309 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.532446 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":43}
I20260807 18:10:59.532872 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.526140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:32851 (local address 127.8.62.130:35423)
0807 18:10:59.526342 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.526349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.526456 (+ 107us) server_negotiation.cc:407] Connection header received
0807 18:10:59.526779 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.526786 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.526877 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.526996 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:59.527469 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.528685 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.530848 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.531722 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.531922 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.531982 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.532357 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.532513 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:10:59.533174 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.526446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:57757 (local address 127.8.62.129:40867)
0807 18:10:59.526879 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.526884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.527253 (+ 369us) server_negotiation.cc:407] Connection header received
0807 18:10:59.527460 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.527466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.527557 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.527717 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:10:59.528169 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.529356 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.531188 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.532053 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.532231 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.532323 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.532739 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.532891 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260807 18:10:59.533282 27853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.526241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:40867 (local address 127.8.62.131:57757)
0807 18:10:59.527133 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0807 18:10:59.527157 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:10:59.527284 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:10:59.527685 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:10:59.527688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:10:59.527708 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:10:59.528011 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.528019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.529530 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.529534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.530993 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:10:59.531007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.532315 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.532322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:10:59.532517 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.532593 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 18:10:59.532708 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 18:10:59.532857 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":7,"thread_start_us":467,"threads_started":1}
I20260807 18:10:59.534557 27640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca84fd77251414785705ea979b1949c" candidate_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3fb93d6af14eceba3464831152cc83" is_pre_election: true
I20260807 18:10:59.534557 27505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca84fd77251414785705ea979b1949c" candidate_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e824c4033248518a357b5e37812561" is_pre_election: true
I20260807 18:10:59.535245 27640 raft_consensus.cc:2454] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97ff6f0c21754d80b2df64ac1a7da13a in term 0.
I20260807 18:10:59.535245 27505 raft_consensus.cc:2454] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97ff6f0c21754d80b2df64ac1a7da13a in term 0.
I20260807 18:10:59.536096 27710 leader_election.cc:302] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 63e824c4033248518a357b5e37812561, 97ff6f0c21754d80b2df64ac1a7da13a, ac3fb93d6af14eceba3464831152cc83; no voters:
I20260807 18:10:59.536700 27851 raft_consensus.cc:2781] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:10:59.536856 27851 raft_consensus.cc:482] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:10:59.536978 27851 raft_consensus.cc:3037] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:59.539466 27851 raft_consensus.cc:504] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.540134 27851 leader_election.cc:288] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [CANDIDATE]: Term 1 election: Requested vote from peers ac3fb93d6af14eceba3464831152cc83 (127.8.62.130:35423), 63e824c4033248518a357b5e37812561 (127.8.62.129:40867)
I20260807 18:10:59.540767 27640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca84fd77251414785705ea979b1949c" candidate_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac3fb93d6af14eceba3464831152cc83"
I20260807 18:10:59.540807 27505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca84fd77251414785705ea979b1949c" candidate_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e824c4033248518a357b5e37812561"
I20260807 18:10:59.540990 27640 raft_consensus.cc:3037] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:59.541097 27505 raft_consensus.cc:3037] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:10:59.543367 27640 raft_consensus.cc:2454] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97ff6f0c21754d80b2df64ac1a7da13a in term 1.
I20260807 18:10:59.543614 27505 raft_consensus.cc:2454] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97ff6f0c21754d80b2df64ac1a7da13a in term 1.
I20260807 18:10:59.543958 27710 leader_election.cc:302] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97ff6f0c21754d80b2df64ac1a7da13a, ac3fb93d6af14eceba3464831152cc83; no voters:
I20260807 18:10:59.544337 27851 raft_consensus.cc:2781] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:10:59.544860 27851 raft_consensus.cc:686] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 1 LEADER]: Becoming Leader. State: Replica: 97ff6f0c21754d80b2df64ac1a7da13a, State: Running, Role: LEADER
I20260807 18:10:59.545435 27851 consensus_queue.cc:237] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } }
I20260807 18:10:59.550168 27358 catalog_manager.cc:5495] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a reported cstate change: term changed from 0 to 1, leader changed from <none> to 97ff6f0c21754d80b2df64ac1a7da13a (127.8.62.131). New cstate: current_term: 1 leader_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97ff6f0c21754d80b2df64ac1a7da13a" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 34285 } health_report { overall_health: HEALTHY } } }
I20260807 18:10:59.607009 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.601329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33004 (local address 127.8.62.131:34285)
0807 18:10:59.601563 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.601572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.601596 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:10:59.601713 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.601717 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.601804 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.601972 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:10:59.602409 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.603164 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.604704 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.605098 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.605248 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.605863 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:10:59.606382 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:10:59.606583 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.606708 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.606757 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:10:59.615619 27640 raft_consensus.cc:1261] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 1 FOLLOWER]: Refusing update from remote peer 97ff6f0c21754d80b2df64ac1a7da13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:59.615916 27505 raft_consensus.cc:1261] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 1 FOLLOWER]: Refusing update from remote peer 97ff6f0c21754d80b2df64ac1a7da13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:10:59.616704 27854 consensus_queue.cc:1035] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac3fb93d6af14eceba3464831152cc83" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:59.617118 27851 consensus_queue.cc:1035] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "63e824c4033248518a357b5e37812561" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 40867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:10:59.630707 27858 mvcc.cc:205] Tried to move back new op lower bound from 7315973159375347712 to 7315973159102095360. Current Snapshot: MvccSnapshot[applied={T|T < 7315973159375347712}]
I20260807 18:10:59.646097 27357 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:59.646530 27357 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:10:59.651069 27357 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e54f5ced4cbf4fa295e1de8238fbc3b4: Sending DeleteTablet for 3 replicas of tablet 9ca84fd77251414785705ea979b1949c
I20260807 18:10:59.652763 27620 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca84fd77251414785705ea979b1949c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:59 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:47394
I20260807 18:10:59.652772 27485 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca84fd77251414785705ea979b1949c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:59 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:34184
I20260807 18:10:59.652992 27757 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca84fd77251414785705ea979b1949c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:10:59 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:32988
I20260807 18:10:59.655121 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27420
I20260807 18:10:59.665328 27868 tablet_replica.cc:330] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: stopping tablet replica
I20260807 18:10:59.665870 27868 raft_consensus.cc:2229] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:59.666455 27868 raft_consensus.cc:2258] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:59.670979 27868 ts_tablet_manager.cc:1905] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:59.673033 27870 tablet_replica.cc:330] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: stopping tablet replica
I20260807 18:10:59.673487 27870 raft_consensus.cc:2229] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:10:59.674230 27870 raft_consensus.cc:2258] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:59.675050 27869 tablet_replica.cc:330] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: stopping tablet replica
I20260807 18:10:59.675520 27869 raft_consensus.cc:2229] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:10:59.676079 27869 raft_consensus.cc:2258] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:10:59.678403 27870 ts_tablet_manager.cc:1905] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:59.678436 27868 ts_tablet_manager.cc:1918] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:59.678589 27868 log.cc:1196] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/9ca84fd77251414785705ea979b1949c
I20260807 18:10:59.679194 27868 ts_tablet_manager.cc:1939] T 9ca84fd77251414785705ea979b1949c P ac3fb93d6af14eceba3464831152cc83: Deleting consensus metadata
I20260807 18:10:59.681262 27342 catalog_manager.cc:4841] TS ac3fb93d6af14eceba3464831152cc83 (127.8.62.130:35423): tablet 9ca84fd77251414785705ea979b1949c (table test-table [id=272d924b51844b1d83b4c1c32a26d1ac]) successfully deleted
Aug 07 18:10:59 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:10:59.683707 27869 ts_tablet_manager.cc:1905] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:10:59.686280 27870 ts_tablet_manager.cc:1918] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:59.686486 27870 log.cc:1196] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/9ca84fd77251414785705ea979b1949c
I20260807 18:10:59.687075 27870 ts_tablet_manager.cc:1939] T 9ca84fd77251414785705ea979b1949c P 97ff6f0c21754d80b2df64ac1a7da13a: Deleting consensus metadata
I20260807 18:10:59.689090 27342 catalog_manager.cc:4841] TS 97ff6f0c21754d80b2df64ac1a7da13a (127.8.62.131:34285): tablet 9ca84fd77251414785705ea979b1949c (table test-table [id=272d924b51844b1d83b4c1c32a26d1ac]) successfully deleted
I20260807 18:10:59.691315 27869 ts_tablet_manager.cc:1918] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:10:59.691510 27869 log.cc:1196] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/9ca84fd77251414785705ea979b1949c
I20260807 18:10:59.692031 27869 ts_tablet_manager.cc:1939] T 9ca84fd77251414785705ea979b1949c P 63e824c4033248518a357b5e37812561: Deleting consensus metadata
I20260807 18:10:59.693872 27841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.657150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34200 (local address 127.8.62.129:40867)
0807 18:10:59.657412 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.657423 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.664077 (+ 6654us) server_negotiation.cc:407] Connection header received
0807 18:10:59.664310 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.664320 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.664455 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.664679 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:59.668266 (+ 3587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.670153 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.673027 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.674014 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.674177 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.684906 (+ 10729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:59.684924 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:59.684945 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:59.685031 (+ 86us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:59.689774 (+ 4743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.690825 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.690835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.690840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.691036 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.691651 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.691660 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.691664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.692206 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:59.692531 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.692957 (+ 426us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.693241 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:10:59.694119 27343 catalog_manager.cc:4841] TS 63e824c4033248518a357b5e37812561 (127.8.62.129:40867): tablet 9ca84fd77251414785705ea979b1949c (table test-table [id=272d924b51844b1d83b4c1c32a26d1ac]) successfully deleted
I20260807 18:10:59.694540 27545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:59.788677 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27420
I20260807 18:10:59.808494 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27556
Aug 07 18:10:59 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:10:59.827200 27840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.809428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.8.62.130:35423)
0807 18:10:59.809631 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.809638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.809662 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:10:59.809759 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.809765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.809861 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.809971 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:59.811325 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.812394 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.813740 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.814416 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.814546 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.819620 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:59.819642 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:59.819666 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:59.819720 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:59.823977 (+ 4257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.824819 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.824830 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.824834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.825036 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.825426 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.825432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.825436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.825977 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:59.826308 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.826508 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.826776 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:10:59.827960 27681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:10:59.920228 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27556
I20260807 18:10:59.940168 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27693
Aug 07 18:10:59 dist-test-slave-rnrw krb5kdc[27311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:10:59.959512 27843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:10:59.941023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33014 (local address 127.8.62.131:34285)
0807 18:10:59.941233 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 18:10:59.941240 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:10:59.941264 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:10:59.941343 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:10:59.941348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:10:59.941470 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:10:59.941620 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:10:59.942947 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.944301 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.946109 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:10:59.946884 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:10:59.947048 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:10:59.952452 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:10:59.952467 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:10:59.952484 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:10:59.952533 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:10:59.955935 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.957145 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.957153 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.957155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.957379 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:10:59.957896 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:10:59.957900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:10:59.957902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:10:59.958304 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:10:59.958597 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 18:10:59.958960 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 18:10:59.959151 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:10:59.960475 27820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:00.089994 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27693
I20260807 18:11:00.111090 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27326
I20260807 18:11:00.112423 27389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:00.227298 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27326
2026-08-07T18:11:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4840 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T18:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:00Z Disabled control of system clock
I20260807 18:11:00.314468 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:41345
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:41345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:11:00.664521 27879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:00.664914 27879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:00.665004 27879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:00.672636 27879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:00.672755 27879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:00.672794 27879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:00.672854 27879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:00.672900 27879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:00.682531 27879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:41345
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:41345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:00.683832 27879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:00.685602 27879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:00.697986 27885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:00.697986 27886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:00.698113 27888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:00.698856 27879 server_base.cc:1034] running on GCE node
I20260807 18:11:00.699738 27879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:00.701301 27879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:00.702543 27879 hybrid_clock.cc:648] HybridClock initialized: now 1786126260702491 us; error 49 us; skew 500 ppm
I20260807 18:11:00.702935 27879 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:00.705417 27879 webserver.cc:460] Webserver started at http://127.8.62.190:46633/ using document root <none> and password file <none>
I20260807 18:11:00.706158 27879 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:00.706302 27879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:00.706641 27879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:00.709842 27879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "5827916b91b742cf8800397e56e2ec57"
format_stamp: "Formatted at 2026-08-07 18:11:00 on dist-test-slave-rnrw"
server_key: "86563454eca9198fadf5f8d4343afb04"
server_key_iv: "f7a61e1bbb11d1185f5d26d0f46d3e9f"
server_key_version: "encryptionkey@0"
I20260807 18:11:00.710733 27879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "5827916b91b742cf8800397e56e2ec57"
format_stamp: "Formatted at 2026-08-07 18:11:00 on dist-test-slave-rnrw"
server_key: "86563454eca9198fadf5f8d4343afb04"
server_key_iv: "f7a61e1bbb11d1185f5d26d0f46d3e9f"
server_key_version: "encryptionkey@0"
I20260807 18:11:00.716109 27879 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:00.719775 27894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:00.721374 27879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:11:00.721567 27879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "5827916b91b742cf8800397e56e2ec57"
format_stamp: "Formatted at 2026-08-07 18:11:00 on dist-test-slave-rnrw"
server_key: "86563454eca9198fadf5f8d4343afb04"
server_key_iv: "f7a61e1bbb11d1185f5d26d0f46d3e9f"
server_key_version: "encryptionkey@0"
I20260807 18:11:00.721809 27879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:00.742497 27879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:00.743491 27879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:00.743808 27879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:00.763597 27879 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:41345
I20260807 18:11:00.763610 27946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:41345 every 8 connection(s)
I20260807 18:11:00.765091 27879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:00.769189 27947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:00.773756 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27879
I20260807 18:11:00.773926 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:00.774413 8442 external_mini_cluster.cc:1428] Setting key ac7c1e7ec68333a587dfd2fe1e10d12e
I20260807 18:11:00.783766 27947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: Bootstrap starting.
I20260807 18:11:00.787657 27947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:00.788952 27947 log.cc:824] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:00.791997 27950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:00.777383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.8.62.190:41345)
0807 18:11:00.779370 (+ 1987us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:00.779391 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:00.779461 (+ 70us) server_negotiation.cc:407] Connection header received
0807 18:11:00.782326 (+ 2865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:00.782359 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:00.782995 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:00.783424 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:00.784058 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:00.785726 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:00.787200 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:00.788049 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:00.788267 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:00.789562 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:00.789612 (+ 50us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:00.789626 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:00.790280 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:00.790360 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:00.790386 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:00.790526 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:00.790742 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 18:11:00.791057 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1096,"thread_start_us":748,"threads_started":1}
I20260807 18:11:00.792295 27947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: No bootstrap required, opened a new log
I20260807 18:11:00.800789 27947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } }
I20260807 18:11:00.801317 27947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:00.801467 27947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5827916b91b742cf8800397e56e2ec57, State: Initialized, Role: FOLLOWER
I20260807 18:11:00.802117 27947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } }
I20260807 18:11:00.802372 27947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:00.802524 27947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:00.802712 27947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:00.805127 27947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } }
I20260807 18:11:00.805714 27947 leader_election.cc:302] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5827916b91b742cf8800397e56e2ec57; no voters:
I20260807 18:11:00.806372 27947 leader_election.cc:288] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:00.806560 27952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:00.807583 27952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [term 1 LEADER]: Becoming Leader. State: Replica: 5827916b91b742cf8800397e56e2ec57, State: Running, Role: LEADER
I20260807 18:11:00.808161 27952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } }
I20260807 18:11:00.808975 27947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:00.811465 27953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5827916b91b742cf8800397e56e2ec57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } } }
I20260807 18:11:00.811976 27953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:00.812572 27954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5827916b91b742cf8800397e56e2ec57. Latest consensus state: current_term: 1 leader_uuid: "5827916b91b742cf8800397e56e2ec57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5827916b91b742cf8800397e56e2ec57" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 41345 } } }
I20260807 18:11:00.816705 27954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:00.818046 27960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:00.823935 27960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:00.833786 27960 catalog_manager.cc:1347] Generated new cluster ID: d577dabd944e42d79655d62d904519c0
I20260807 18:11:00.833927 27960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:00.849792 27960 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:00.851456 27960 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:00.874684 27960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: Generated new TSK 0
I20260807 18:11:00.875743 27960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:00.879909 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--builtin_ntp_servers=127.8.62.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:11:01.271759 27972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:01.272127 27972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:01.272228 27972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:01.279687 27972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:01.279810 27972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:01.279922 27972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:11:01.289430 27972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:01.290949 27972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:01.293048 27972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:01.307196 27977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:01.307165 27978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:01.307384 27980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:01.308126 27972 server_base.cc:1034] running on GCE node
I20260807 18:11:01.308866 27972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:01.309998 27972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:01.311223 27972 hybrid_clock.cc:648] HybridClock initialized: now 1786126261311162 us; error 53 us; skew 500 ppm
I20260807 18:11:01.311715 27972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:01.314306 27972 webserver.cc:460] Webserver started at http://127.8.62.129:39095/ using document root <none> and password file <none>
I20260807 18:11:01.315025 27972 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:01.315119 27972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:01.315403 27972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:01.318591 27972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "91789ad5d91742d5b0bfa062204d63fb"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "5a00a0effa512564686df5ccd5e8c74b"
server_key_iv: "ce1e6bf8e4542ec73e1ec40d60a17525"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.319394 27972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "91789ad5d91742d5b0bfa062204d63fb"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "5a00a0effa512564686df5ccd5e8c74b"
server_key_iv: "ce1e6bf8e4542ec73e1ec40d60a17525"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.325400 27972 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.002s
I20260807 18:11:01.329412 27986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.330917 27972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:11:01.331071 27972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "91789ad5d91742d5b0bfa062204d63fb"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "5a00a0effa512564686df5ccd5e8c74b"
server_key_iv: "ce1e6bf8e4542ec73e1ec40d60a17525"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.331270 27972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:01.348171 27972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:01.349136 27972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:01.349423 27972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:01.350459 27972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:01.352094 27972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:01.352193 27972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.352267 27972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:01.352324 27972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.387921 27972 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:36027
I20260807 18:11:01.388046 28101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:36027 every 8 connection(s)
I20260807 18:11:01.389434 27972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:11:01.394562 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 27972
I20260807 18:11:01.394758 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:11:01.395211 8442 external_mini_cluster.cc:1428] Setting key 702a8ac5d07b0f4e4247dfe6ffc2ed61
I20260807 18:11:01.398078 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--builtin_ntp_servers=127.8.62.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:11:01.402864 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:01.392378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:36313 (local address 127.8.62.190:41345)
0807 18:11:01.392884 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:01.392889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:01.394160 (+ 1271us) server_negotiation.cc:407] Connection header received
0807 18:11:01.396147 (+ 1987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:01.396153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:01.396291 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:01.396426 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:01.397615 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.398391 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.399959 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.400369 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.400508 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:01.401975 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:01.401979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:01.401982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:01.402002 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:01.402044 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:01.402056 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:01.402158 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:01.402387 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 18:11:01.402491 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":284,"threads_started":1}
I20260807 18:11:01.403891 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:01.393047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41345 (local address 127.8.62.129:36313)
0807 18:11:01.393990 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:01.394027 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:01.395856 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:01.396737 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:01.396746 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:01.396842 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:01.397435 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:01.397452 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.398555 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.398559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:01.399813 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:01.399826 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.400629 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.400638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:01.400886 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:01.401637 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:01.401640 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:01.401682 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:01.401764 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:01.402156 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:01.402169 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:01.402180 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:11:01.402364 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 18:11:01.402792 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":298,"threads_started":1}
I20260807 18:11:01.405630 28102 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41345
I20260807 18:11:01.405946 28102 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:01.406733 28102 heartbeater.cc:511] Master 127.8.62.190:41345 requested a full tablet report, sending...
I20260807 18:11:01.409173 27912 ts_manager.cc:194] Registered new tserver with Master: 91789ad5d91742d5b0bfa062204d63fb (127.8.62.129:36027)
I20260807 18:11:01.411491 27912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.129:36313
W20260807 18:11:01.751353 28106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:01.751675 28106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:01.751767 28106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:01.759202 28106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:01.759317 28106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:01.759428 28106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:11:01.772680 28106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:01.774408 28106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:01.776742 28106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:01.792992 28111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:01.792924 28112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:01.792975 28114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:01.793879 28106 server_base.cc:1034] running on GCE node
I20260807 18:11:01.794667 28106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:01.795802 28106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:01.797051 28106 hybrid_clock.cc:648] HybridClock initialized: now 1786126261796992 us; error 58 us; skew 500 ppm
I20260807 18:11:01.797408 28106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:01.799918 28106 webserver.cc:460] Webserver started at http://127.8.62.130:42375/ using document root <none> and password file <none>
I20260807 18:11:01.800841 28106 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:01.800983 28106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:01.801326 28106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:01.804366 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "dc2f6f365c514fcfb978fa06aae73cb3"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "92f4e1f316eda6bed2670b68363f3ba3"
server_key_iv: "49275c617ba98ee74ede42d32c2ed026"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.805305 28106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "dc2f6f365c514fcfb978fa06aae73cb3"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "92f4e1f316eda6bed2670b68363f3ba3"
server_key_iv: "49275c617ba98ee74ede42d32c2ed026"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.810925 28106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:01.814785 28120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.816231 28106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:11:01.816437 28106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "dc2f6f365c514fcfb978fa06aae73cb3"
format_stamp: "Formatted at 2026-08-07 18:11:01 on dist-test-slave-rnrw"
server_key: "92f4e1f316eda6bed2670b68363f3ba3"
server_key_iv: "49275c617ba98ee74ede42d32c2ed026"
server_key_version: "encryptionkey@0"
I20260807 18:11:01.816784 28106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:01.839444 28106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:01.840401 28106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:01.840782 28106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:01.841871 28106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:01.843542 28106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:01.843748 28106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.843906 28106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:01.844009 28106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:01.879383 28106 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:40231
I20260807 18:11:01.879518 28236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:40231 every 8 connection(s)
I20260807 18:11:01.880890 28106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:11:01.889891 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28106
I20260807 18:11:01.890110 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:11:01.890583 8442 external_mini_cluster.cc:1428] Setting key b8decbd93cc78c94f84d21421c151189
I20260807 18:11:01.893591 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--builtin_ntp_servers=127.8.62.148:33525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:11:01.895017 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:01.883642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:33697 (local address 127.8.62.190:41345)
0807 18:11:01.883832 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:01.883837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:01.885459 (+ 1622us) server_negotiation.cc:407] Connection header received
0807 18:11:01.887385 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:01.887390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:01.887505 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:01.887624 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:01.888866 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.889734 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.891722 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.892228 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.892383 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:01.894109 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:01.894114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:01.894117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:01.894137 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:01.894171 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:01.894184 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:01.894286 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:01.894538 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 18:11:01.894645 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":71}
I20260807 18:11:01.896193 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:01.884186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41345 (local address 127.8.62.130:33697)
0807 18:11:01.885278 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:01.885323 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:01.887080 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:01.887872 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:01.887881 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:01.887983 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:01.888686 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:01.888708 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.889929 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.889933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:01.891536 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:01.891551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:01.892584 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:01.892594 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:01.892888 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:01.893675 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:01.893684 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:01.893759 (+ 75us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:01.893884 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:01.894286 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:01.894300 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:01.894311 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:11:01.894517 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 18:11:01.894970 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":397,"threads_started":1}
I20260807 18:11:01.897943 28237 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41345
I20260807 18:11:01.898316 28237 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:01.899317 28237 heartbeater.cc:511] Master 127.8.62.190:41345 requested a full tablet report, sending...
I20260807 18:11:01.901285 27912 ts_manager.cc:194] Registered new tserver with Master: dc2f6f365c514fcfb978fa06aae73cb3 (127.8.62.130:40231)
I20260807 18:11:01.902556 27912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.130:33697
W20260807 18:11:02.244237 28240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:02.244624 28240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:02.244756 28240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:02.252331 28240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:02.252449 28240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:02.252643 28240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:11:02.262233 28240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:33525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:41345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:02.263571 28240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:02.265460 28240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:02.280336 28245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:02.280429 28246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:02.280388 28248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:02.281571 28240 server_base.cc:1034] running on GCE node
I20260807 18:11:02.282341 28240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:02.283418 28240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:02.284729 28240 hybrid_clock.cc:648] HybridClock initialized: now 1786126262284669 us; error 45 us; skew 500 ppm
I20260807 18:11:02.285063 28240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:02.287719 28240 webserver.cc:460] Webserver started at http://127.8.62.131:35363/ using document root <none> and password file <none>
I20260807 18:11:02.288496 28240 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:02.288677 28240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:02.289001 28240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:02.292200 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "eb3cd0ffc7034b3084420ccaac76119b"
format_stamp: "Formatted at 2026-08-07 18:11:02 on dist-test-slave-rnrw"
server_key: "17d6075d9954ddec3452f89b568ff559"
server_key_iv: "dc60be29bc2f629839595b553ae3f72f"
server_key_version: "encryptionkey@0"
I20260807 18:11:02.293113 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "eb3cd0ffc7034b3084420ccaac76119b"
format_stamp: "Formatted at 2026-08-07 18:11:02 on dist-test-slave-rnrw"
server_key: "17d6075d9954ddec3452f89b568ff559"
server_key_iv: "dc60be29bc2f629839595b553ae3f72f"
server_key_version: "encryptionkey@0"
I20260807 18:11:02.298935 28240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:02.303175 28254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:02.304920 28240 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:11:02.305083 28240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "eb3cd0ffc7034b3084420ccaac76119b"
format_stamp: "Formatted at 2026-08-07 18:11:02 on dist-test-slave-rnrw"
server_key: "17d6075d9954ddec3452f89b568ff559"
server_key_iv: "dc60be29bc2f629839595b553ae3f72f"
server_key_version: "encryptionkey@0"
I20260807 18:11:02.305301 28240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:02.350276 28240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:02.351198 28240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:02.351495 28240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:02.352535 28240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:02.354149 28240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:02.354246 28240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:02.354331 28240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:02.354393 28240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:02.392895 28240 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:35269
I20260807 18:11:02.392935 28371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:35269 every 8 connection(s)
I20260807 18:11:02.394415 28240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:11:02.394831 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28240
I20260807 18:11:02.395035 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:11:02.395646 8442 external_mini_cluster.cc:1428] Setting key 3dfc2d77b37ef7c61e78d2b17ca5df73
I20260807 18:11:02.409413 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.397913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:44973 (local address 127.8.62.190:41345)
0807 18:11:02.398620 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.398625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.400081 (+ 1456us) server_negotiation.cc:407] Connection header received
0807 18:11:02.402338 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.402344 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.402453 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.402585 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:02.403738 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.404515 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.406209 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.406663 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.406814 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.408337 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:02.408342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:02.408345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:02.408368 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:02.408421 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:02.408434 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:02.408539 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.408842 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.408960 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":484,"threads_started":1}
I20260807 18:11:02.410751 28375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.398666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:41345 (local address 127.8.62.131:44973)
0807 18:11:02.399890 (+ 1224us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.399928 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.402049 (+ 2121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.402851 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.402860 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.402964 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:02.403562 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.403578 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.404723 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.404729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.406036 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.406049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.406922 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.406929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.407192 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.407954 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:02.407956 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:02.408010 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:02.408102 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:02.408538 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:02.408588 (+ 50us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:02.408599 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.408828 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.409323 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":357,"threads_started":1}
I20260807 18:11:02.412284 28372 heartbeater.cc:347] Connected to a master server at 127.8.62.190:41345
I20260807 18:11:02.412688 28372 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:02.413555 28372 heartbeater.cc:511] Master 127.8.62.190:41345 requested a full tablet report, sending...
I20260807 18:11:02.414737 28102 heartbeater.cc:503] Master 127.8.62.190:41345 was elected leader, sending a full tablet report...
I20260807 18:11:02.415308 27912 ts_manager.cc:194] Registered new tserver with Master: eb3cd0ffc7034b3084420ccaac76119b (127.8.62.131:35269)
I20260807 18:11:02.416680 27912 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.131:44973
I20260807 18:11:02.425622 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:11:02.437662 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.430128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33794 (local address 127.8.62.190:41345)
0807 18:11:02.430389 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.430400 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.430980 (+ 580us) server_negotiation.cc:407] Connection header received
0807 18:11:02.431167 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.431171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.431239 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.431332 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:02.431873 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.432851 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.434644 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.435155 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.435295 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.436831 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:02.436838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:02.436843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:02.436874 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:02.436908 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:02.436922 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:02.437115 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.437273 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.437338 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:11:02.442200 27912 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:11:02.449893 27912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:11:02.487679 28388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.464162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:36027 (local address 127.0.0.1:60052)
0807 18:11:02.476674 (+ 12512us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.476723 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.476939 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.478250 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.478257 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.478291 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:02.478812 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.478831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.481097 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.481102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.483680 (+ 2578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.483705 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.486470 (+ 2765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.486478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.486708 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.486837 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.486983 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.487209 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12221,"mutex_wait_us":35,"thread_start_us":11122,"threads_started":1}
I20260807 18:11:02.488024 28389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.463744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:40231 (local address 127.0.0.1:53108)
0807 18:11:02.477287 (+ 13543us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.477307 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.477417 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.478700 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.478705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.478739 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:02.479243 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.479260 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.482556 (+ 3296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.482563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.485204 (+ 2641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.485221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.487226 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.487232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.487444 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.487485 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.487551 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.487706 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11738,"thread_start_us":12489,"threads_started":1}
I20260807 18:11:02.488039 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.464902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60052 (local address 127.8.62.129:36027)
0807 18:11:02.475204 (+ 10302us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.475218 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.477190 (+ 1972us) server_negotiation.cc:407] Connection header received
0807 18:11:02.477321 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.477334 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.477682 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.477904 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:02.479107 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.480913 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.483979 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.485838 (+ 1859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.485990 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.486320 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.487397 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.487627 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10075,"thread_start_us":9899,"threads_started":1}
I20260807 18:11:02.489701 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.463772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53108 (local address 127.8.62.130:40231)
0807 18:11:02.477987 (+ 14215us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.478001 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.478033 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:11:02.478154 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.478164 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.478437 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.479502 (+ 1065us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:02.479766 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.482392 (+ 2626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.485439 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.486989 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.488704 (+ 1715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.488992 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.489073 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.489275 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14023,"thread_start_us":13900,"threads_started":1}
I20260807 18:11:02.490540 28392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.478145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35269 (local address 127.0.0.1:44714)
0807 18:11:02.482889 (+ 4744us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.482904 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.483085 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.483669 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.483673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.483703 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:02.484155 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.484164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.485845 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.485850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.488058 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.488074 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.489810 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.489821 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.490028 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.490067 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.490162 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.490265 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4631,"mutex_wait_us":1,"thread_start_us":3459,"threads_started":1}
I20260807 18:11:02.491447 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.477282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44714 (local address 127.8.62.131:35269)
0807 18:11:02.481490 (+ 4208us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.481501 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.483024 (+ 1523us) server_negotiation.cc:407] Connection header received
0807 18:11:02.483270 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.483277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.483475 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.483688 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:02.484328 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.485664 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.488310 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.489575 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.489740 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.490897 (+ 1157us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.490973 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.491152 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4015,"thread_start_us":3842,"threads_started":1}
I20260807 18:11:02.492762 28037 tablet_service.cc:1467] Processing CreateTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 (DEFAULT_TABLE table=test-table [id=f423635a2ef24ea09823a4c5a272eb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:02.493485 28172 tablet_service.cc:1467] Processing CreateTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 (DEFAULT_TABLE table=test-table [id=f423635a2ef24ea09823a4c5a272eb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:02.494479 28037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aff5e2c9894e80bf3bea1ce808a193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:02.494874 28172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aff5e2c9894e80bf3bea1ce808a193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:02.495095 28302 tablet_service.cc:1467] Processing CreateTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 (DEFAULT_TABLE table=test-table [id=f423635a2ef24ea09823a4c5a272eb5e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:02.496378 28302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48aff5e2c9894e80bf3bea1ce808a193. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:02.511101 28393 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Bootstrap starting.
I20260807 18:11:02.512241 28394 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Bootstrap starting.
I20260807 18:11:02.515686 28393 tablet_bootstrap.cc:654] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:02.516144 28395 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Bootstrap starting.
I20260807 18:11:02.516412 28394 tablet_bootstrap.cc:654] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:02.517362 28393 log.cc:824] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:02.517875 28394 log.cc:824] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:02.519865 28395 tablet_bootstrap.cc:654] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:02.521219 28395 log.cc:824] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:02.521320 28393 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: No bootstrap required, opened a new log
I20260807 18:11:02.521447 28394 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: No bootstrap required, opened a new log
I20260807 18:11:02.521656 28393 ts_tablet_manager.cc:1397] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260807 18:11:02.521724 28394 ts_tablet_manager.cc:1397] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:11:02.529750 28395 tablet_bootstrap.cc:492] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: No bootstrap required, opened a new log
I20260807 18:11:02.530077 28395 ts_tablet_manager.cc:1397] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260807 18:11:02.531083 28393 raft_consensus.cc:348] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.531505 28393 raft_consensus.cc:374] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:02.531626 28393 raft_consensus.cc:729] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb3cd0ffc7034b3084420ccaac76119b, State: Initialized, Role: FOLLOWER
I20260807 18:11:02.532317 28393 consensus_queue.cc:260] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.533555 28372 heartbeater.cc:503] Master 127.8.62.190:41345 was elected leader, sending a full tablet report...
I20260807 18:11:02.533381 28394 raft_consensus.cc:348] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.533844 28394 raft_consensus.cc:374] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:02.533936 28394 raft_consensus.cc:729] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91789ad5d91742d5b0bfa062204d63fb, State: Initialized, Role: FOLLOWER
I20260807 18:11:02.534466 28393 ts_tablet_manager.cc:1428] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Time spent starting tablet: real 0.013s user 0.007s sys 0.005s
I20260807 18:11:02.534690 28394 consensus_queue.cc:260] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.537127 28394 ts_tablet_manager.cc:1428] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Time spent starting tablet: real 0.015s user 0.004s sys 0.010s
I20260807 18:11:02.540907 28395 raft_consensus.cc:348] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.541445 28395 raft_consensus.cc:374] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:02.541548 28395 raft_consensus.cc:729] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc2f6f365c514fcfb978fa06aae73cb3, State: Initialized, Role: FOLLOWER
I20260807 18:11:02.542142 28395 consensus_queue.cc:260] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.543275 28237 heartbeater.cc:503] Master 127.8.62.190:41345 was elected leader, sending a full tablet report...
I20260807 18:11:02.543759 28395 ts_tablet_manager.cc:1428] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260807 18:11:02.633140 28238 tablet.cc:2369] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:02.642266 28103 tablet.cc:2369] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:02.646755 28373 tablet.cc:2369] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:02.687023 28401 raft_consensus.cc:482] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:11:02.687453 28401 raft_consensus.cc:504] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.689747 28401 leader_election.cc:288] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb3cd0ffc7034b3084420ccaac76119b (127.8.62.131:35269), 91789ad5d91742d5b0bfa062204d63fb (127.8.62.129:36027)
I20260807 18:11:02.696681 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.690196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:58435 (local address 127.8.62.129:36027)
0807 18:11:02.690419 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.690431 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.691176 (+ 745us) server_negotiation.cc:407] Connection header received
0807 18:11:02.691422 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.691426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.691501 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.691604 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:02.692169 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.692951 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.694887 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.695701 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.695903 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.695977 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.696259 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.696395 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 18:11:02.696943 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.690283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:58583 (local address 127.8.62.131:35269)
0807 18:11:02.690507 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.690513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.691464 (+ 951us) server_negotiation.cc:407] Connection header received
0807 18:11:02.691683 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.691687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.691748 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.691851 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:02.692444 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.693232 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.695008 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.695991 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.696164 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.696236 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.696532 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.696702 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:11:02.697058 28403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.690196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:35269 (local address 127.8.62.130:58583)
0807 18:11:02.691341 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.691356 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.691518 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.691868 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.691872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.691899 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:02.692282 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.692291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.693383 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.693389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.694809 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.694822 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.696168 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.696174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.696352 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.696375 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.696480 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.696651 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"mutex_wait_us":249,"thread_start_us":523,"threads_started":1}
I20260807 18:11:02.697058 28402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.690198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:36027 (local address 127.8.62.130:58435)
0807 18:11:02.691015 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:02.691037 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:02.691257 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:02.691622 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:02.691627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:02.691652 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:02.692012 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.692021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.693096 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.693103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.694678 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:02.694692 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.695909 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.695915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:02.696082 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.696107 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:11:02.696226 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:11:02.696378 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":435,"threads_started":1}
I20260807 18:11:02.698138 28324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aff5e2c9894e80bf3bea1ce808a193" candidate_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb3cd0ffc7034b3084420ccaac76119b" is_pre_election: true
I20260807 18:11:02.698140 28057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aff5e2c9894e80bf3bea1ce808a193" candidate_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91789ad5d91742d5b0bfa062204d63fb" is_pre_election: true
I20260807 18:11:02.698663 28324 raft_consensus.cc:2454] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc2f6f365c514fcfb978fa06aae73cb3 in term 0.
I20260807 18:11:02.698680 28057 raft_consensus.cc:2454] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc2f6f365c514fcfb978fa06aae73cb3 in term 0.
I20260807 18:11:02.699417 28124 leader_election.cc:302] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 91789ad5d91742d5b0bfa062204d63fb, dc2f6f365c514fcfb978fa06aae73cb3, eb3cd0ffc7034b3084420ccaac76119b; no voters:
I20260807 18:11:02.699838 28401 raft_consensus.cc:2781] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:11:02.700002 28401 raft_consensus.cc:482] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:11:02.700131 28401 raft_consensus.cc:3037] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:02.702529 28401 raft_consensus.cc:504] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.703233 28401 leader_election.cc:288] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [CANDIDATE]: Term 1 election: Requested vote from peers eb3cd0ffc7034b3084420ccaac76119b (127.8.62.131:35269), 91789ad5d91742d5b0bfa062204d63fb (127.8.62.129:36027)
I20260807 18:11:02.703874 28324 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aff5e2c9894e80bf3bea1ce808a193" candidate_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb3cd0ffc7034b3084420ccaac76119b"
I20260807 18:11:02.703891 28057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48aff5e2c9894e80bf3bea1ce808a193" candidate_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91789ad5d91742d5b0bfa062204d63fb"
I20260807 18:11:02.704128 28324 raft_consensus.cc:3037] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:02.704138 28057 raft_consensus.cc:3037] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:02.706573 28324 raft_consensus.cc:2454] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc2f6f365c514fcfb978fa06aae73cb3 in term 1.
I20260807 18:11:02.706593 28057 raft_consensus.cc:2454] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc2f6f365c514fcfb978fa06aae73cb3 in term 1.
I20260807 18:11:02.707139 28124 leader_election.cc:302] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc2f6f365c514fcfb978fa06aae73cb3, eb3cd0ffc7034b3084420ccaac76119b; no voters:
I20260807 18:11:02.707533 28401 raft_consensus.cc:2781] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:02.708031 28401 raft_consensus.cc:686] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 1 LEADER]: Becoming Leader. State: Replica: dc2f6f365c514fcfb978fa06aae73cb3, State: Running, Role: LEADER
I20260807 18:11:02.708593 28401 consensus_queue.cc:237] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } }
I20260807 18:11:02.714067 27911 catalog_manager.cc:5495] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc2f6f365c514fcfb978fa06aae73cb3 (127.8.62.130). New cstate: current_term: 1 leader_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc2f6f365c514fcfb978fa06aae73cb3" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 40231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 } health_report { overall_health: UNKNOWN } } }
I20260807 18:11:02.788535 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.781131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.8.62.130:40231)
0807 18:11:02.781438 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.781444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.781472 (+ 28us) server_negotiation.cc:407] Connection header received
0807 18:11:02.781713 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.781718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.781812 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.782004 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:02.782654 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.783895 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.786214 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.786639 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.786741 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.787319 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:02.787867 (+ 548us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:02.788022 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.788147 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.788187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260807 18:11:02.798134 28057 raft_consensus.cc:1261] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 1 FOLLOWER]: Refusing update from remote peer dc2f6f365c514fcfb978fa06aae73cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:02.798134 28324 raft_consensus.cc:1261] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 1 FOLLOWER]: Refusing update from remote peer dc2f6f365c514fcfb978fa06aae73cb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:02.799469 28404 consensus_queue.cc:1035] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb3cd0ffc7034b3084420ccaac76119b" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:02.799901 28401 consensus_queue.cc:1035] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91789ad5d91742d5b0bfa062204d63fb" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 36027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:02.810030 28407 mvcc.cc:205] Tried to move back new op lower bound from 7315973172408193024 to 7315973172059095040. Current Snapshot: MvccSnapshot[applied={T|T < 7315973172408193024}]
I20260807 18:11:02.811863 28409 mvcc.cc:205] Tried to move back new op lower bound from 7315973172408193024 to 7315973172059095040. Current Snapshot: MvccSnapshot[applied={T|T < 7315973172408193024}]
I20260807 18:11:02.819099 28408 mvcc.cc:205] Tried to move back new op lower bound from 7315973172408193024 to 7315973172059095040. Current Snapshot: MvccSnapshot[applied={T|T < 7315973172408193024}]
I20260807 18:11:02.830304 27911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:02.830680 27911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:02.835124 27911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5827916b91b742cf8800397e56e2ec57: Sending DeleteTablet for 3 replicas of tablet 48aff5e2c9894e80bf3bea1ce808a193
I20260807 18:11:02.836719 28172 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:02 UTC) from {username='slave'} at 127.0.0.1:53108
I20260807 18:11:02.836897 28302 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:02 UTC) from {username='slave'} at 127.0.0.1:44714
I20260807 18:11:02.837531 28037 tablet_service.cc:1514] Processing DeleteTablet for tablet 48aff5e2c9894e80bf3bea1ce808a193 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:02 UTC) from {username='slave'} at 127.0.0.1:60052
I20260807 18:11:02.837997 28417 tablet_replica.cc:330] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: stopping tablet replica
I20260807 18:11:02.838487 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27972
I20260807 18:11:02.838619 28417 raft_consensus.cc:2229] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:11:02.839498 28417 raft_consensus.cc:2258] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:02.842356 28417 ts_tablet_manager.cc:1905] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:02.849218 28418 tablet_replica.cc:330] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: stopping tablet replica
I20260807 18:11:02.849808 28418 raft_consensus.cc:2229] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:02.850409 28418 raft_consensus.cc:2258] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:02.851095 28419 tablet_replica.cc:330] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: stopping tablet replica
I20260807 18:11:02.851712 28419 raft_consensus.cc:2229] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:02.852305 28419 raft_consensus.cc:2258] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:02.853935 28418 ts_tablet_manager.cc:1905] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:02.854104 28417 ts_tablet_manager.cc:1918] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:02.854301 28417 log.cc:1196] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/48aff5e2c9894e80bf3bea1ce808a193
I20260807 18:11:02.854661 28419 ts_tablet_manager.cc:1905] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:02.854920 28417 ts_tablet_manager.cc:1939] T 48aff5e2c9894e80bf3bea1ce808a193 P dc2f6f365c514fcfb978fa06aae73cb3: Deleting consensus metadata
I20260807 18:11:02.857030 27897 catalog_manager.cc:4841] TS dc2f6f365c514fcfb978fa06aae73cb3 (127.8.62.130:40231): tablet 48aff5e2c9894e80bf3bea1ce808a193 (table test-table [id=f423635a2ef24ea09823a4c5a272eb5e]) successfully deleted
I20260807 18:11:02.861430 28390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.840850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.8.62.129:36027)
0807 18:11:02.841065 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.841072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.845856 (+ 4784us) server_negotiation.cc:407] Connection header received
0807 18:11:02.846071 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.846080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.846241 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.846373 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:02.847278 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.849058 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.856803 (+ 7745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.857550 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.857672 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.859436 (+ 1764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:02.859448 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:02.859473 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:02.860374 (+ 901us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:02.860446 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:02.860483 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:02.860667 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.860909 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.861036 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:11:02.862083 28419 ts_tablet_manager.cc:1918] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:02.862262 28419 log.cc:1196] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/48aff5e2c9894e80bf3bea1ce808a193
I20260807 18:11:02.862867 28419 ts_tablet_manager.cc:1939] T 48aff5e2c9894e80bf3bea1ce808a193 P 91789ad5d91742d5b0bfa062204d63fb: Deleting consensus metadata
I20260807 18:11:02.863230 28097 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:02.864277 28418 ts_tablet_manager.cc:1918] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:02.864600 28418 log.cc:1196] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/48aff5e2c9894e80bf3bea1ce808a193
I20260807 18:11:02.865267 28418 ts_tablet_manager.cc:1939] T 48aff5e2c9894e80bf3bea1ce808a193 P eb3cd0ffc7034b3084420ccaac76119b: Deleting consensus metadata
I20260807 18:11:02.867972 27897 catalog_manager.cc:4841] TS eb3cd0ffc7034b3084420ccaac76119b (127.8.62.131:35269): tablet 48aff5e2c9894e80bf3bea1ce808a193 (table test-table [id=f423635a2ef24ea09823a4c5a272eb5e]) successfully deleted
I20260807 18:11:02.958010 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27972
I20260807 18:11:02.958236 27895 catalog_manager.cc:4841] TS 91789ad5d91742d5b0bfa062204d63fb (127.8.62.129:36027): tablet 48aff5e2c9894e80bf3bea1ce808a193 (table test-table [id=f423635a2ef24ea09823a4c5a272eb5e]) successfully deleted
I20260807 18:11:02.977718 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28106
I20260807 18:11:02.985913 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:02.978541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.8.62.130:40231)
0807 18:11:02.978739 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:02.978745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:02.978791 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:11:02.978949 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:02.978955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:02.979086 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:02.979195 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:02.979575 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.980813 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.982090 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:02.982763 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:02.982904 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:02.984006 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:02.984019 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:02.984040 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:02.985033 (+ 993us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:02.985098 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:02.985131 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:02.985249 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:02.985402 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 18:11:02.985512 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 18:11:02.986862 28232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:03.106534 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28106
I20260807 18:11:03.124166 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28240
I20260807 18:11:03.131705 28391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:03.125020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.8.62.131:35269)
0807 18:11:03.125216 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:03.125220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:03.125241 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:11:03.125480 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:03.125484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:03.125615 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:03.125760 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:03.126430 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:03.127225 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:03.128660 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:03.129052 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:03.129153 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:03.130279 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:03.130289 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:03.130304 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:03.130957 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:03.131010 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:03.131032 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:03.131126 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:03.131322 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 18:11:03.131440 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:11:03.132396 28366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:03.223507 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28240
I20260807 18:11:03.242161 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 27879
I20260807 18:11:03.243441 27942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:03.357854 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 27879
2026-08-07T18:11:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3129 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:03 dist-test-slave-rnrw krb5kdc[28427](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:03 dist-test-slave-rnrw krb5kdc[28427](info): set up 2 sockets
Aug 07 18:11:03 dist-test-slave-rnrw krb5kdc[28427](info): commencing operation
krb5kdc: starting...
W20260807 18:11:04.524791 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:04 dist-test-slave-rnrw krb5kdc[28427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:04.823899 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46813
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:05.187559 28440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:05.187980 28440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:05.188169 28440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:05.195849 28440 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:05.196058 28440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:05.196164 28440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:05.196255 28440 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:05.196349 28440 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:05.207353 28440 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:46813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:46813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:05.208967 28440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:05.210439 28440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:05.223780 28445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:05.223744 28448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:05.223706 28446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:05.224632 28440 server_base.cc:1034] running on GCE node
I20260807 18:11:05.225610 28440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:05.227263 28440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:05.228609 28440 hybrid_clock.cc:648] HybridClock initialized: now 1786126265228448 us; error 81 us; skew 500 ppm
I20260807 18:11:05.229000 28440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:05 dist-test-slave-rnrw krb5kdc[28427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:05.236088 28440 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:05.238070 28440 webserver.cc:460] Webserver started at http://127.8.62.190:44875/ using document root <none> and password file <none>
I20260807 18:11:05.238870 28440 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:05.239038 28440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:05.239372 28440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:05.242650 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "2d1573eb3a914d769bb88631f9080d2f"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "811ea1cb338e3018628917240a4b9980"
server_key_iv: "a4ae8dcfba92dc5eb9b3d1bb88df57db"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.243511 28440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "2d1573eb3a914d769bb88631f9080d2f"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "811ea1cb338e3018628917240a4b9980"
server_key_iv: "a4ae8dcfba92dc5eb9b3d1bb88df57db"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.249372 28440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:11:05.253400 28455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:05.255064 28440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 18:11:05.255297 28440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "2d1573eb3a914d769bb88631f9080d2f"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "811ea1cb338e3018628917240a4b9980"
server_key_iv: "a4ae8dcfba92dc5eb9b3d1bb88df57db"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.255548 28440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:05.292625 28440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:05.297374 28440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:05.297709 28440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:05.317987 28440 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:46813
I20260807 18:11:05.318041 28509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:46813 every 8 connection(s)
I20260807 18:11:05.319547 28440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:05.323879 28510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:05.324429 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28440
I20260807 18:11:05.324679 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:05.325176 8442 external_mini_cluster.cc:1428] Setting key ab348be119a41a3248a33d0e2061b3aa
I20260807 18:11:05.339558 28510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: Bootstrap starting.
Aug 07 18:11:05 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:05.343309 28510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:05.344439 28510 log.cc:824] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:05.347999 28510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: No bootstrap required, opened a new log
I20260807 18:11:05.351991 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:05.327254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34044 (local address 127.8.62.190:46813)
0807 18:11:05.328228 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:05.328240 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:05.328284 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:11:05.330493 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:05.330524 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:05.330949 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:05.331313 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:05.333323 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.334613 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.336613 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.337403 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.337560 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:05.342927 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:05.342937 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:05.342953 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:05.342998 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:05.346936 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:05.347773 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:05.347782 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:05.347790 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:05.348013 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:05.348480 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:05.348485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:05.348492 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:05.349273 (+ 781us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:05.349576 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:05.349966 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 18:11:05.350474 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":433,"threads_started":1}
I20260807 18:11:05.357807 28510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } }
I20260807 18:11:05.358174 28510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:05.358296 28510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d1573eb3a914d769bb88631f9080d2f, State: Initialized, Role: FOLLOWER
I20260807 18:11:05.359074 28510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } }
I20260807 18:11:05.359366 28510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:05.359632 28510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:05.359820 28510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:05.363323 28510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } }
I20260807 18:11:05.363821 28510 leader_election.cc:302] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d1573eb3a914d769bb88631f9080d2f; no voters:
I20260807 18:11:05.364389 28510 leader_election.cc:288] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:05.364568 28515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:05.366639 28510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:05.366864 28515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [term 1 LEADER]: Becoming Leader. State: Replica: 2d1573eb3a914d769bb88631f9080d2f, State: Running, Role: LEADER
I20260807 18:11:05.367444 28515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } }
I20260807 18:11:05.370644 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d1573eb3a914d769bb88631f9080d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } } }
I20260807 18:11:05.371240 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:05.370915 28517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d1573eb3a914d769bb88631f9080d2f. Latest consensus state: current_term: 1 leader_uuid: "2d1573eb3a914d769bb88631f9080d2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d1573eb3a914d769bb88631f9080d2f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 46813 } } }
I20260807 18:11:05.371448 28517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:05.373442 28526 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:05.379658 28526 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:05.389986 28526 catalog_manager.cc:1347] Generated new cluster ID: 87fc3d69107a41c393063006b85f5073
I20260807 18:11:05.390168 28526 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:05.402406 28526 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:05.404178 28526 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:05.416764 28526 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: Generated new TSK 0
I20260807 18:11:05.417793 28526 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.62.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
Entry for principal kudu/127.8.62.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab.
I20260807 18:11:05.492200 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:05.849362 28536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:05.849668 28536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:05.849763 28536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:05.857663 28536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:05.857793 28536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:05.857908 28536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:11:05.868695 28536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.129.keytab
--principal=kudu/127.8.62.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:05.870167 28536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:05.871636 28536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:05.886289 28541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:05.886195 28542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:05.886230 28544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:05.887162 28536 server_base.cc:1034] running on GCE node
I20260807 18:11:05.887964 28536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:05.889137 28536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:05.890378 28536 hybrid_clock.cc:648] HybridClock initialized: now 1786126265890325 us; error 38 us; skew 500 ppm
I20260807 18:11:05.890810 28536 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:05 dist-test-slave-rnrw krb5kdc[28427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:05.894438 28536 init.cc:377] Logged in from keytab as kudu/127.8.62.129@KRBTEST.COM (short username kudu)
I20260807 18:11:05.896032 28536 webserver.cc:460] Webserver started at http://127.8.62.129:42341/ using document root <none> and password file <none>
I20260807 18:11:05.896924 28536 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:05.897086 28536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:05.897425 28536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:05.900655 28536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "2d0fc243146545828b5aa82b179615d8"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "5527c5c4fca9072cffdc268f9ac342c7"
server_key_iv: "8e12046c60a0d1d959739455369bc1b2"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.901559 28536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "2d0fc243146545828b5aa82b179615d8"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "5527c5c4fca9072cffdc268f9ac342c7"
server_key_iv: "8e12046c60a0d1d959739455369bc1b2"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.907522 28536 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:05.911556 28551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:05.913290 28536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:05.913497 28536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "2d0fc243146545828b5aa82b179615d8"
format_stamp: "Formatted at 2026-08-07 18:11:05 on dist-test-slave-rnrw"
server_key: "5527c5c4fca9072cffdc268f9ac342c7"
server_key_iv: "8e12046c60a0d1d959739455369bc1b2"
server_key_version: "encryptionkey@0"
I20260807 18:11:05.913725 28536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:05.939072 28536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:05.943799 28536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:05.944141 28536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:05.945307 28536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:05.946743 28536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:05.946849 28536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:05.947005 28536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:05.947081 28536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:05.983132 28536 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:34403
I20260807 18:11:05.983223 28666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:34403 every 8 connection(s)
I20260807 18:11:05.984495 28536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:11:05.991288 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28536
I20260807 18:11:05.991511 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:11:05.991961 8442 external_mini_cluster.cc:1428] Setting key 7f0defeed6832d06d5f60ca5b0e968ed
Aug 07 18:11:05 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126265, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.129@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:06.008435 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:05.987225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:34617 (local address 127.8.62.190:46813)
0807 18:11:05.987732 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:05.987739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:05.988830 (+ 1091us) server_negotiation.cc:407] Connection header received
0807 18:11:05.990802 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:05.990810 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:05.990935 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:05.991081 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:05.993045 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.993936 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.995671 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.996445 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.996645 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:06.000401 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:06.000411 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:06.000419 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:06.000475 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:06.004344 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:06.005606 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:06.005611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:06.005613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:06.005766 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:06.006434 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:06.006438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:06.006440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:06.006761 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:06.006893 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:06.007971 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0807 18:11:06.008163 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":239,"threads_started":1}
I20260807 18:11:06.010386 28670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:05.987849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46813 (local address 127.8.62.129:34617)
0807 18:11:05.988648 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:05.988697 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:05.990471 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:05.991333 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:05.991342 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:05.992043 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:05.992858 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:05.992881 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.994120 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.994124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:05.995493 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:05.995507 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:05.996770 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:05.996775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:05.997003 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:05.997752 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:05.997778 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:06.000110 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:06.004537 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:06.004576 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:06.004602 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:06.005437 (+ 835us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:06.005925 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:06.005954 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:06.005962 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:06.006295 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:06.006922 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:06.006943 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:06.007502 (+ 559us) client_negotiation.cc:769] Sending connection context
0807 18:11:06.007864 (+ 362us) client_negotiation.cc:240] Negotiation successful
0807 18:11:06.008589 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":292,"threads_started":1}
I20260807 18:11:06.012418 28667 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46813
I20260807 18:11:06.012833 28667 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:06.013849 28667 heartbeater.cc:511] Master 127.8.62.190:46813 requested a full tablet report, sending...
I20260807 18:11:06.016817 28472 ts_manager.cc:194] Registered new tserver with Master: 2d0fc243146545828b5aa82b179615d8 (127.8.62.129:34403)
I20260807 18:11:06.019476 28472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.129@KRBTEST.COM'} at 127.8.62.129:34617
WARNING: no policy specified for kudu/127.8.62.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
Entry for principal kudu/127.8.62.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab.
I20260807 18:11:06.062228 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:06.410881 28673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:06.411190 28673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:06.411350 28673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:06.419281 28673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:06.419453 28673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:06.419639 28673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:11:06.430114 28673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.130.keytab
--principal=kudu/127.8.62.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:06.431571 28673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:06.433104 28673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:06.447628 28678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:06.447719 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:06.448474 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:06.449262 28673 server_base.cc:1034] running on GCE node
I20260807 18:11:06.449970 28673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:06.450995 28673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:06.452242 28673 hybrid_clock.cc:648] HybridClock initialized: now 1786126266452161 us; error 62 us; skew 500 ppm
I20260807 18:11:06.452824 28673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:06 dist-test-slave-rnrw krb5kdc[28427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:06.456244 28673 init.cc:377] Logged in from keytab as kudu/127.8.62.130@KRBTEST.COM (short username kudu)
I20260807 18:11:06.457926 28673 webserver.cc:460] Webserver started at http://127.8.62.130:44315/ using document root <none> and password file <none>
I20260807 18:11:06.458658 28673 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:06.458762 28673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:06.459039 28673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:06.462240 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "273c078e3f966034b91673c79c7f0af7"
server_key_iv: "09ed3492818451d6e6cbe89eaab613fc"
server_key_version: "encryptionkey@0"
I20260807 18:11:06.463017 28673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "273c078e3f966034b91673c79c7f0af7"
server_key_iv: "09ed3492818451d6e6cbe89eaab613fc"
server_key_version: "encryptionkey@0"
I20260807 18:11:06.468689 28673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:11:06.472607 28688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:06.474164 28673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260807 18:11:06.474313 28673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "273c078e3f966034b91673c79c7f0af7"
server_key_iv: "09ed3492818451d6e6cbe89eaab613fc"
server_key_version: "encryptionkey@0"
I20260807 18:11:06.474506 28673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:06.486714 28673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:06.491272 28673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:06.491541 28673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:06.492520 28673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:06.493927 28673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:06.494002 28673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:06.494089 28673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:06.494134 28673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:06.529775 28673 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:35517
I20260807 18:11:06.529798 28803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:35517 every 8 connection(s)
I20260807 18:11:06.531183 28673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:11:06.532156 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28673
I20260807 18:11:06.532359 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:11:06.533039 8442 external_mini_cluster.cc:1428] Setting key 0d162da415bc4a1e933c59edb65520dd
Aug 07 18:11:06 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.130@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:06.555565 28806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:06.534420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:45339 (local address 127.8.62.190:46813)
0807 18:11:06.534964 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:06.534971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:06.536674 (+ 1703us) server_negotiation.cc:407] Connection header received
0807 18:11:06.539119 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:06.539124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:06.539191 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:06.539287 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:06.541566 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:06.542368 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:06.544958 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:06.545392 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:06.545493 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:06.549602 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:06.549607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:06.549610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:06.549643 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:06.552284 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:06.553121 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:06.553125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:06.553127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:06.553260 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:06.553726 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:06.553730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:06.553732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:06.554001 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:06.554162 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:06.554890 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 18:11:06.555153 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":304,"threads_started":1}
I20260807 18:11:06.556677 28807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:06.535056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46813 (local address 127.8.62.130:45339)
0807 18:11:06.535937 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:06.535997 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:06.538820 (+ 2823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:06.539597 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:06.539612 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:06.540457 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:06.541344 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:06.541375 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:06.542545 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:06.542553 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:06.544805 (+ 2252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:06.544824 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:06.545629 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:06.545639 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:06.545982 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:06.546812 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:06.546836 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:06.549333 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:06.552432 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:06.552441 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:06.552458 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:06.552967 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:06.553377 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:06.553383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:06.553387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:06.553591 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:06.554132 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:06.554141 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:06.554513 (+ 372us) client_negotiation.cc:769] Sending connection context
0807 18:11:06.554792 (+ 279us) client_negotiation.cc:240] Negotiation successful
0807 18:11:06.555381 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":343,"threads_started":1}
I20260807 18:11:06.558364 28804 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46813
I20260807 18:11:06.558719 28804 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:06.559616 28804 heartbeater.cc:511] Master 127.8.62.190:46813 requested a full tablet report, sending...
I20260807 18:11:06.561235 28472 ts_manager.cc:194] Registered new tserver with Master: bb8b5d96ad45425a8f0ddbf4d2b8141c (127.8.62.130:35517)
I20260807 18:11:06.562072 28472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.130@KRBTEST.COM'} at 127.8.62.130:45339
WARNING: no policy specified for kudu/127.8.62.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
Entry for principal kudu/127.8.62.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab.
I20260807 18:11:06.599797 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:45723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:06.940740 28810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:06.941033 28810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:06.941133 28810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:06.948356 28810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:06.948479 28810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:06.948668 28810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:11:06.958384 28810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:45723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.131.keytab
--principal=kudu/127.8.62.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:46813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:06.959762 28810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:06.961248 28810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:06.975072 28815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:06.975076 28818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:06.975006 28816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:06.975522 28810 server_base.cc:1034] running on GCE node
I20260807 18:11:06.976257 28810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:06.977257 28810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:06.978550 28810 hybrid_clock.cc:648] HybridClock initialized: now 1786126266978486 us; error 54 us; skew 500 ppm
I20260807 18:11:06.978896 28810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:06 dist-test-slave-rnrw krb5kdc[28427](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:06.982246 28810 init.cc:377] Logged in from keytab as kudu/127.8.62.131@KRBTEST.COM (short username kudu)
I20260807 18:11:06.984169 28810 webserver.cc:460] Webserver started at http://127.8.62.131:39471/ using document root <none> and password file <none>
I20260807 18:11:06.985072 28810 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:06.985214 28810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:06.985581 28810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:06.989567 28810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "e0b3362da2c443159aad6120c40f5651"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "1ec8575658588e816b174ddfcce06f38"
server_key_iv: "6867d6b8ff81218b3861cdec4bf7c286"
server_key_version: "encryptionkey@0"
I20260807 18:11:06.990299 28810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "e0b3362da2c443159aad6120c40f5651"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "1ec8575658588e816b174ddfcce06f38"
server_key_iv: "6867d6b8ff81218b3861cdec4bf7c286"
server_key_version: "encryptionkey@0"
I20260807 18:11:06.995541 28810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:11:06.999290 28825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:07.000960 28810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:07.001117 28810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "e0b3362da2c443159aad6120c40f5651"
format_stamp: "Formatted at 2026-08-07 18:11:06 on dist-test-slave-rnrw"
server_key: "1ec8575658588e816b174ddfcce06f38"
server_key_iv: "6867d6b8ff81218b3861cdec4bf7c286"
server_key_version: "encryptionkey@0"
I20260807 18:11:07.001292 28810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:07.016856 28810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:07.021400 28810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:07.021731 28810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:07.022814 28810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:07.024133 28667 heartbeater.cc:503] Master 127.8.62.190:46813 was elected leader, sending a full tablet report...
I20260807 18:11:07.024296 28810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:07.024397 28810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:07.024545 28810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:07.024629 28810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 18:11:07.058441 28810 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:43305
I20260807 18:11:07.058465 28938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:43305 every 8 connection(s)
I20260807 18:11:07.059859 28810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:11:07.067412 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28810
I20260807 18:11:07.067662 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:11:07.068276 8442 external_mini_cluster.cc:1428] Setting key 34e27d7c7272a4ab413d67f5e6ca4512
Aug 07 18:11:07 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126266, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.131@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:07.082043 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.062720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:34091 (local address 127.8.62.190:46813)
0807 18:11:07.063213 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.063218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.064189 (+ 971us) server_negotiation.cc:407] Connection header received
0807 18:11:07.066114 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.066118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.066185 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.066282 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:07.068026 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.069159 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.071491 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.072133 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.072250 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.076094 (+ 3844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:07.076100 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:07.076103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:07.076142 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:07.078845 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.079656 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.079660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.079662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.079792 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.080229 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.080234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.080236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.080613 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:07.080768 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.081424 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.081666 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":247,"threads_started":1}
I20260807 18:11:07.083004 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.063282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:46813 (local address 127.8.62.131:34091)
0807 18:11:07.064017 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.064063 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.065829 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.066484 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.066491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.067153 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:07.067851 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.067871 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.069320 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.069325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.071336 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.071350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.072370 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.072376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.072695 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.073478 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:07.073503 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:07.075876 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:07.078991 (+ 3115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:07.078998 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:07.079011 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:07.079549 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:07.079889 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:07.079893 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:07.079895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:07.080089 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:07.080748 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:07.080754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:07.081072 (+ 318us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.081307 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.081739 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":310,"threads_started":1}
I20260807 18:11:07.084599 28939 heartbeater.cc:347] Connected to a master server at 127.8.62.190:46813
I20260807 18:11:07.085001 28939 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:07.085755 28939 heartbeater.cc:511] Master 127.8.62.190:46813 requested a full tablet report, sending...
I20260807 18:11:07.087332 28472 ts_manager.cc:194] Registered new tserver with Master: e0b3362da2c443159aad6120c40f5651 (127.8.62.131:43305)
I20260807 18:11:07.088260 28472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.62.131@KRBTEST.COM'} at 127.8.62.131:34091
I20260807 18:11:07.096938 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:11:07.114444 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.101093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.8.62.190:46813)
0807 18:11:07.101280 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.101285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.101781 (+ 496us) server_negotiation.cc:407] Connection header received
0807 18:11:07.101920 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.101924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.101989 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.102077 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:07.103599 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.104328 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.105747 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.106211 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.106377 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.108883 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:07.108893 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:07.108897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:07.108931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:07.111542 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.112282 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.112290 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.112294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.112438 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.112857 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.112865 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.112868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.113222 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:07.113483 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.113865 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.114124 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 18:11:07.118767 28472 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:11:07.125845 28472 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:11:07.162740 28952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.139358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35517 (local address 127.0.0.1:37280)
0807 18:11:07.152100 (+ 12742us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.152162 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.152423 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.154194 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.154198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.154219 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:07.154726 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.154738 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.156709 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.156714 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.159089 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.159104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.161272 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.161278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.161871 (+ 593us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.161943 (+ 72us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.162089 (+ 146us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.162321 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12369,"thread_start_us":12199,"threads_started":1}
I20260807 18:11:07.162727 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.139723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.8.62.130:35517)
0807 18:11:07.153035 (+ 13312us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.153046 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.153080 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:11:07.153346 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.153358 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.153714 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.153967 (+ 253us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:07.154935 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.156473 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.159272 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.160654 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.160836 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.161075 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.162177 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.162375 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13134,"thread_start_us":12968,"threads_started":1}
I20260807 18:11:07.164093 28953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.139434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:43305 (local address 127.0.0.1:39776)
0807 18:11:07.152282 (+ 12848us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.152299 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.153022 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.153933 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.153941 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.153991 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:07.154448 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.154468 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.158148 (+ 3680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.158154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.160611 (+ 2457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.160627 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.163099 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.163106 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.163336 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.163375 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.163497 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.163642 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12705,"mutex_wait_us":4,"thread_start_us":13018,"threads_started":1}
I20260807 18:11:07.164134 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.139999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.8.62.131:43305)
0807 18:11:07.150990 (+ 10991us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.151006 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.152417 (+ 1411us) server_negotiation.cc:407] Connection header received
0807 18:11:07.153299 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.153309 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.153626 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.153841 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:07.154755 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.157015 (+ 2260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.160844 (+ 3829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.162544 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.162677 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.162940 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.163577 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.163759 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10689,"thread_start_us":11043,"threads_started":1}
I20260807 18:11:07.167567 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.153244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45836 (local address 127.8.62.129:34403)
0807 18:11:07.157827 (+ 4583us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.157840 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.158594 (+ 754us) server_negotiation.cc:407] Connection header received
0807 18:11:07.158724 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.158733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.159051 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.160029 (+ 978us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:07.160294 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.161940 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.164816 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.166302 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.166456 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.166702 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.167040 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.167215 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4385,"thread_start_us":4193,"threads_started":1}
I20260807 18:11:07.167589 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.154490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45836)
0807 18:11:07.158235 (+ 3745us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.158257 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.158411 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.159298 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.159303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.159342 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:07.159793 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.159805 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.162130 (+ 2325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.162136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.164519 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.164532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.166612 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.166619 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.166819 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.166856 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.166963 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.167103 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3598,"mutex_wait_us":55,"thread_start_us":3357,"threads_started":1}
I20260807 18:11:07.167869 28737 tablet_service.cc:1467] Processing CreateTablet for tablet 34556afa4ea24b18bacb13e99036403f (DEFAULT_TABLE table=test-table [id=0766334d17d041708fde4f9127d46c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:07.168753 28873 tablet_service.cc:1467] Processing CreateTablet for tablet 34556afa4ea24b18bacb13e99036403f (DEFAULT_TABLE table=test-table [id=0766334d17d041708fde4f9127d46c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:07.169834 28737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34556afa4ea24b18bacb13e99036403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:07.170603 28873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34556afa4ea24b18bacb13e99036403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:07.172191 28599 tablet_service.cc:1467] Processing CreateTablet for tablet 34556afa4ea24b18bacb13e99036403f (DEFAULT_TABLE table=test-table [id=0766334d17d041708fde4f9127d46c24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:07.173889 28599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34556afa4ea24b18bacb13e99036403f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:07.183141 28958 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Bootstrap starting.
I20260807 18:11:07.186924 28959 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Bootstrap starting.
I20260807 18:11:07.187904 28958 tablet_bootstrap.cc:654] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:07.189385 28958 log.cc:824] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:07.191141 28959 tablet_bootstrap.cc:654] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:07.192423 28960 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Bootstrap starting.
I20260807 18:11:07.192730 28959 log.cc:824] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:07.194640 28958 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: No bootstrap required, opened a new log
I20260807 18:11:07.194976 28958 ts_tablet_manager.cc:1397] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 18:11:07.196584 28960 tablet_bootstrap.cc:654] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:07.197512 28959 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: No bootstrap required, opened a new log
I20260807 18:11:07.197805 28959 ts_tablet_manager.cc:1397] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 18:11:07.197970 28960 log.cc:824] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:07.201435 28960 tablet_bootstrap.cc:492] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: No bootstrap required, opened a new log
I20260807 18:11:07.201746 28960 ts_tablet_manager.cc:1397] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 18:11:07.203424 28958 raft_consensus.cc:348] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.203823 28958 raft_consensus.cc:374] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:07.203924 28958 raft_consensus.cc:729] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8b5d96ad45425a8f0ddbf4d2b8141c, State: Initialized, Role: FOLLOWER
I20260807 18:11:07.204628 28958 consensus_queue.cc:260] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.205873 28804 heartbeater.cc:503] Master 127.8.62.190:46813 was elected leader, sending a full tablet report...
I20260807 18:11:07.206610 28958 ts_tablet_manager.cc:1428] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Time spent starting tablet: real 0.011s user 0.013s sys 0.000s
I20260807 18:11:07.210418 28959 raft_consensus.cc:348] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.210830 28959 raft_consensus.cc:374] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:07.210955 28959 raft_consensus.cc:729] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b3362da2c443159aad6120c40f5651, State: Initialized, Role: FOLLOWER
I20260807 18:11:07.211585 28959 consensus_queue.cc:260] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.212775 28939 heartbeater.cc:503] Master 127.8.62.190:46813 was elected leader, sending a full tablet report...
I20260807 18:11:07.213371 28959 ts_tablet_manager.cc:1428] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260807 18:11:07.212280 28960 raft_consensus.cc:348] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.213904 28960 raft_consensus.cc:374] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:07.214030 28960 raft_consensus.cc:729] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0fc243146545828b5aa82b179615d8, State: Initialized, Role: FOLLOWER
I20260807 18:11:07.214603 28960 consensus_queue.cc:260] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.216672 28960 ts_tablet_manager.cc:1428] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260807 18:11:07.236956 28668 tablet.cc:2369] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:07.283509 28805 tablet.cc:2369] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:07.311808 28940 tablet.cc:2369] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:07.336458 28965 raft_consensus.cc:482] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:11:07.336800 28965 raft_consensus.cc:504] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.338454 28965 leader_election.cc:288] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb8b5d96ad45425a8f0ddbf4d2b8141c (127.8.62.130:35517), 2d0fc243146545828b5aa82b179615d8 (127.8.62.129:34403)
I20260807 18:11:07.346897 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.339145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:34265 (local address 127.8.62.129:34403)
0807 18:11:07.339403 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.339412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.339987 (+ 575us) server_negotiation.cc:407] Connection header received
0807 18:11:07.340157 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.340161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.340220 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.340294 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:07.340863 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.341825 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.344563 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.345893 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.346053 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.346117 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.346526 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.346655 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:11:07.346894 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.338958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:39141 (local address 127.8.62.130:35517)
0807 18:11:07.339197 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.339204 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.339502 (+ 298us) server_negotiation.cc:407] Connection header received
0807 18:11:07.339765 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.339773 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.339854 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.339962 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:07.340389 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.341498 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.344462 (+ 2964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.345742 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.345911 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.346012 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.346471 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.346615 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:11:07.347151 28942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.338972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:35517 (local address 127.8.62.131:39141)
0807 18:11:07.339355 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.339393 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.339572 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.339995 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.340001 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.340023 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:07.340221 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.340229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.341664 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.341669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.344232 (+ 2563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.344252 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.345934 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.345940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.346212 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.346255 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.346383 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.346672 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":119}
I20260807 18:11:07.347208 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.339075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.8.62.131:34265)
0807 18:11:07.339864 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.339877 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:07.340016 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:07.340336 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:07.340341 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:07.340371 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:07.340693 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.340701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.342022 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.342029 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.344355 (+ 2326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:07.344371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.346097 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.346104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:07.346275 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.346313 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 18:11:07.346403 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 18:11:07.346672 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":81,"thread_start_us":452,"threads_started":1}
I20260807 18:11:07.348636 28758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34556afa4ea24b18bacb13e99036403f" candidate_uuid: "e0b3362da2c443159aad6120c40f5651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" is_pre_election: true
I20260807 18:11:07.348625 28620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34556afa4ea24b18bacb13e99036403f" candidate_uuid: "e0b3362da2c443159aad6120c40f5651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d0fc243146545828b5aa82b179615d8" is_pre_election: true
I20260807 18:11:07.349224 28758 raft_consensus.cc:2454] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0b3362da2c443159aad6120c40f5651 in term 0.
I20260807 18:11:07.349267 28620 raft_consensus.cc:2454] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0b3362da2c443159aad6120c40f5651 in term 0.
I20260807 18:11:07.350010 28828 leader_election.cc:302] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d0fc243146545828b5aa82b179615d8, bb8b5d96ad45425a8f0ddbf4d2b8141c, e0b3362da2c443159aad6120c40f5651; no voters:
I20260807 18:11:07.350493 28965 raft_consensus.cc:2781] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:11:07.350626 28965 raft_consensus.cc:482] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:11:07.350770 28965 raft_consensus.cc:3037] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:07.353230 28965 raft_consensus.cc:504] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.353905 28965 leader_election.cc:288] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [CANDIDATE]: Term 1 election: Requested vote from peers bb8b5d96ad45425a8f0ddbf4d2b8141c (127.8.62.130:35517), 2d0fc243146545828b5aa82b179615d8 (127.8.62.129:34403)
I20260807 18:11:07.354405 28758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34556afa4ea24b18bacb13e99036403f" candidate_uuid: "e0b3362da2c443159aad6120c40f5651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c"
I20260807 18:11:07.354539 28620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34556afa4ea24b18bacb13e99036403f" candidate_uuid: "e0b3362da2c443159aad6120c40f5651" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d0fc243146545828b5aa82b179615d8"
I20260807 18:11:07.354628 28758 raft_consensus.cc:3037] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:07.354787 28620 raft_consensus.cc:3037] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:07.357007 28758 raft_consensus.cc:2454] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0b3362da2c443159aad6120c40f5651 in term 1.
I20260807 18:11:07.357092 28620 raft_consensus.cc:2454] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0b3362da2c443159aad6120c40f5651 in term 1.
I20260807 18:11:07.357537 28826 leader_election.cc:302] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb8b5d96ad45425a8f0ddbf4d2b8141c, e0b3362da2c443159aad6120c40f5651; no voters:
I20260807 18:11:07.357870 28965 raft_consensus.cc:2781] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:07.358346 28965 raft_consensus.cc:686] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 1 LEADER]: Becoming Leader. State: Replica: e0b3362da2c443159aad6120c40f5651, State: Running, Role: LEADER
I20260807 18:11:07.358827 28965 consensus_queue.cc:237] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } }
I20260807 18:11:07.363710 28472 catalog_manager.cc:5495] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 reported cstate change: term changed from 0 to 1, leader changed from <none> to e0b3362da2c443159aad6120c40f5651 (127.8.62.131). New cstate: current_term: 1 leader_uuid: "e0b3362da2c443159aad6120c40f5651" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0b3362da2c443159aad6120c40f5651" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 43305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 } health_report { overall_health: UNKNOWN } } }
I20260807 18:11:07.406628 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.401442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.8.62.131:43305)
0807 18:11:07.401692 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.401697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.401716 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:11:07.401909 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.401913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.401988 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.402115 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:07.402427 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.403179 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.404600 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.404990 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.405082 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.405532 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:07.406039 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:07.406153 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.406250 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.406312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:11:07.414817 28758 raft_consensus.cc:1261] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 1 FOLLOWER]: Refusing update from remote peer e0b3362da2c443159aad6120c40f5651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:07.414816 28620 raft_consensus.cc:1261] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 1 FOLLOWER]: Refusing update from remote peer e0b3362da2c443159aad6120c40f5651: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:07.415809 28968 consensus_queue.cc:1035] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb8b5d96ad45425a8f0ddbf4d2b8141c" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 35517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:07.416179 28965 consensus_queue.cc:1035] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d0fc243146545828b5aa82b179615d8" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 34403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:07.427141 28971 mvcc.cc:205] Tried to move back new op lower bound from 7315973191320993792 to 7315973191105499136. Current Snapshot: MvccSnapshot[applied={T|T < 7315973191320993792}]
I20260807 18:11:07.428292 28972 mvcc.cc:205] Tried to move back new op lower bound from 7315973191320993792 to 7315973191105499136. Current Snapshot: MvccSnapshot[applied={T|T < 7315973191320993792}]
I20260807 18:11:07.446735 28472 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:07.447139 28472 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:07.453838 28472 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d1573eb3a914d769bb88631f9080d2f: Sending DeleteTablet for 3 replicas of tablet 34556afa4ea24b18bacb13e99036403f
I20260807 18:11:07.455945 28737 tablet_service.cc:1514] Processing DeleteTablet for tablet 34556afa4ea24b18bacb13e99036403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:07 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:37280
I20260807 18:11:07.456621 28599 tablet_service.cc:1514] Processing DeleteTablet for tablet 34556afa4ea24b18bacb13e99036403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:07 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:45836
I20260807 18:11:07.458339 28873 tablet_service.cc:1514] Processing DeleteTablet for tablet 34556afa4ea24b18bacb13e99036403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:07 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:39776
I20260807 18:11:07.459203 28598 tablet_service.cc:1514] Processing DeleteTablet for tablet 34556afa4ea24b18bacb13e99036403f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:07 UTC) from {username='kudu', principal='kudu/127.8.62.190@KRBTEST.COM'} at 127.0.0.1:45836
I20260807 18:11:07.462961 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28536
I20260807 18:11:07.468849 28984 tablet_replica.cc:330] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: stopping tablet replica
I20260807 18:11:07.469555 28984 raft_consensus.cc:2229] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:11:07.471302 28982 tablet_replica.cc:330] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: stopping tablet replica
I20260807 18:11:07.471995 28982 raft_consensus.cc:2229] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:07.472889 28983 tablet_replica.cc:330] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: stopping tablet replica
I20260807 18:11:07.474292 28982 raft_consensus.cc:2258] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:07.474969 28983 raft_consensus.cc:2229] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:07.478865 28983 raft_consensus.cc:2258] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:07.480612 28982 ts_tablet_manager.cc:1905] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:07.481961 28984 raft_consensus.cc:2258] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 07 18:11:07 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.129@KRBTEST.COM
I20260807 18:11:07.484488 28984 ts_tablet_manager.cc:1905] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:07.490660 28982 ts_tablet_manager.cc:1918] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:07.490976 28982 log.cc:1196] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/34556afa4ea24b18bacb13e99036403f
I20260807 18:11:07.491712 28982 ts_tablet_manager.cc:1939] T 34556afa4ea24b18bacb13e99036403f P bb8b5d96ad45425a8f0ddbf4d2b8141c: Deleting consensus metadata
I20260807 18:11:07.493498 28983 ts_tablet_manager.cc:1905] T 34556afa4ea24b18bacb13e99036403f P 2d0fc243146545828b5aa82b179615d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:07.494114 28456 catalog_manager.cc:4841] TS bb8b5d96ad45425a8f0ddbf4d2b8141c (127.8.62.130:35517): tablet 34556afa4ea24b18bacb13e99036403f (table test-table [id=0766334d17d041708fde4f9127d46c24]) successfully deleted
I20260807 18:11:07.494522 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.464043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45852 (local address 127.8.62.129:34403)
0807 18:11:07.464230 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.464238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.467557 (+ 3319us) server_negotiation.cc:407] Connection header received
0807 18:11:07.470294 (+ 2737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.470300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.470401 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.470507 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:07.475964 (+ 5457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.477223 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.479434 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.480126 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.480224 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.486548 (+ 6324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:07.486566 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:07.486584 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:07.486636 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:07.490584 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.491554 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.491564 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.491567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.491758 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.492271 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.492277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.492280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.492821 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:07.493166 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.493812 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.494086 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:11:07.496279 28984 ts_tablet_manager.cc:1918] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:07.496532 28984 log.cc:1196] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/34556afa4ea24b18bacb13e99036403f
I20260807 18:11:07.496867 28662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:07.497277 28984 ts_tablet_manager.cc:1939] T 34556afa4ea24b18bacb13e99036403f P e0b3362da2c443159aad6120c40f5651: Deleting consensus metadata
I20260807 18:11:07.499379 28458 catalog_manager.cc:4841] TS e0b3362da2c443159aad6120c40f5651 (127.8.62.131:43305): tablet 34556afa4ea24b18bacb13e99036403f (table test-table [id=0766334d17d041708fde4f9127d46c24]) successfully deleted
I20260807 18:11:07.597922 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28536
W20260807 18:11:07.625623 28458 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.62.129:34403 (error 108)
I20260807 18:11:07.626464 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28673
I20260807 18:11:07.627743 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.627124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45864)
0807 18:11:07.627449 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.627570 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
W20260807 18:11:07.628082 28458 catalog_manager.cc:4584] TS 2d0fc243146545828b5aa82b179615d8 (127.8.62.129:34403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 34556afa4ea24b18bacb13e99036403f: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Aug 07 18:11:07 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.130@KRBTEST.COM
I20260807 18:11:07.645272 28954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.627361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.8.62.130:35517)
0807 18:11:07.627563 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.627568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.627657 (+ 89us) server_negotiation.cc:407] Connection header received
0807 18:11:07.627876 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.627883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.627993 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.628122 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:07.629985 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.630776 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.632853 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.633264 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.633364 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.638808 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:07.638826 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:07.638843 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:07.638890 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:07.642037 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.642847 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.642855 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.642857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.642992 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.643414 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.643423 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.643425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.643887 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:07.644211 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.644659 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.644912 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:11:07.645993 28799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:07.673099 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.672471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45868)
0807 18:11:07.672781 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.672909 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260807 18:11:07.674302 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.673888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45880)
0807 18:11:07.674054 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.674160 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 18:11:07.681746 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.681262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45892)
0807 18:11:07.681465 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.681570 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:11:07.682906 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.682536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45902)
0807 18:11:07.682692 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.682773 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 18:11:07.723675 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.723063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45906)
0807 18:11:07.723333 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.723450 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 18:11:07.725128 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.724616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45914)
0807 18:11:07.724856 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.724958 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 18:11:07.752235 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28673
I20260807 18:11:07.759682 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.759204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45926)
0807 18:11:07.759407 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.759503 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:11:07.760897 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.760505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45936)
0807 18:11:07.760707 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.760789 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 18:11:07.777948 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28810
Aug 07 18:11:07 dist-test-slave-rnrw krb5kdc[28427](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.131@KRBTEST.COM
I20260807 18:11:07.796469 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.778771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.8.62.131:43305)
0807 18:11:07.778937 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:07.778941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:07.778964 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:11:07.779204 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:07.779208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:07.779272 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:07.779354 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:07.781226 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.782087 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.783969 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:07.784427 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:07.784518 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:07.790132 (+ 5614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:07.790147 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:07.790163 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:07.790214 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:07.793390 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.794147 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.794155 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.794157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.794331 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:07.794742 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:07.794747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:07.794757 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:07.795150 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:07.795575 (+ 425us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:07.795842 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 18:11:07.796065 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:07.797299 28934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:07.826452 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.825950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45944)
0807 18:11:07.826169 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.826273 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 18:11:07.827633 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.827225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45948)
0807 18:11:07.827385 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.827472 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 18:11:07.862315 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.861773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45964)
0807 18:11:07.862022 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.862116 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:11:07.863638 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:07.863195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45974)
0807 18:11:07.863394 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:07.863499 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 18:11:07.924166 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28810
I20260807 18:11:07.946602 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28440
I20260807 18:11:07.947839 28505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:08.062417 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:08.061841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:34403 (local address 127.0.0.1:45978)
0807 18:11:08.062059 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:08.062170 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.129:34403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 18:11:08.062438 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28440
2026-08-07T18:11:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4704 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T18:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:08Z Disabled control of system clock
I20260807 18:11:08.150594 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37295
--webserver_interface=127.8.62.190
--webserver_port=0
--builtin_ntp_servers=127.8.62.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:37295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:11:08.505968 28994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:08.506292 28994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:08.506456 28994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:08.514020 28994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:08.514140 28994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:08.514225 28994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:08.514320 28994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:08.514403 28994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:08.523880 28994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:37295
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:08.525278 28994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:08.527086 28994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:08.539208 29000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:08.539240 29002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:08.539305 28999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:08.540622 28994 server_base.cc:1034] running on GCE node
I20260807 18:11:08.541572 28994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:08.543334 28994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:08.544673 28994 hybrid_clock.cc:648] HybridClock initialized: now 1786126268544633 us; error 34 us; skew 500 ppm
I20260807 18:11:08.545086 28994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:08.547749 28994 webserver.cc:460] Webserver started at http://127.8.62.190:33503/ using document root <none> and password file <none>
I20260807 18:11:08.548491 28994 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:08.548679 28994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:08.548980 28994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:08.551954 28994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "22b8a1f2db1942038acb0742e612d182"
format_stamp: "Formatted at 2026-08-07 18:11:08 on dist-test-slave-rnrw"
server_key: "356983011032bd66eb6edcedfffd1b4d"
server_key_iv: "23ace137704134a0c9372658d42ad198"
server_key_version: "encryptionkey@0"
I20260807 18:11:08.552822 28994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "22b8a1f2db1942038acb0742e612d182"
format_stamp: "Formatted at 2026-08-07 18:11:08 on dist-test-slave-rnrw"
server_key: "356983011032bd66eb6edcedfffd1b4d"
server_key_iv: "23ace137704134a0c9372658d42ad198"
server_key_version: "encryptionkey@0"
I20260807 18:11:08.558064 28994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 18:11:08.561685 29008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:08.563050 28994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 18:11:08.563249 28994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "22b8a1f2db1942038acb0742e612d182"
format_stamp: "Formatted at 2026-08-07 18:11:08 on dist-test-slave-rnrw"
server_key: "356983011032bd66eb6edcedfffd1b4d"
server_key_iv: "23ace137704134a0c9372658d42ad198"
server_key_version: "encryptionkey@0"
I20260807 18:11:08.563473 28994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:08.588263 28994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:08.589263 28994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:08.589591 28994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:08.610519 28994 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:37295
I20260807 18:11:08.610581 29060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:37295 every 8 connection(s)
I20260807 18:11:08.612028 28994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:08.616230 29061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:08.619521 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 28994
I20260807 18:11:08.619707 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:08.620263 8442 external_mini_cluster.cc:1428] Setting key 1f43a92b3a18974cc144f6c7d5d73167
I20260807 18:11:08.629215 29061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: Bootstrap starting.
I20260807 18:11:08.632485 29061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:08.633737 29061 log.cc:824] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:08.635046 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:08.622539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.8.62.190:37295)
0807 18:11:08.623615 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:08.623626 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:08.623672 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:11:08.625527 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:08.625557 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:08.626101 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:08.626557 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:08.627188 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:08.628517 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:08.630033 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:08.630776 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:08.630981 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:08.632213 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:08.632222 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:08.632236 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:08.632924 (+ 688us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:08.633031 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:08.633054 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:08.633208 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:08.633431 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 18:11:08.633763 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":366,"threads_started":1}
I20260807 18:11:08.636977 29061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: No bootstrap required, opened a new log
I20260807 18:11:08.645280 29061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } }
I20260807 18:11:08.645748 29061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:08.645872 29061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22b8a1f2db1942038acb0742e612d182, State: Initialized, Role: FOLLOWER
I20260807 18:11:08.646631 29061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } }
I20260807 18:11:08.646974 29061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:08.647123 29061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:08.647327 29061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:08.649730 29061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } }
I20260807 18:11:08.650295 29061 leader_election.cc:302] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22b8a1f2db1942038acb0742e612d182; no voters:
I20260807 18:11:08.650938 29061 leader_election.cc:288] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:08.651173 29066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:08.652101 29066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [term 1 LEADER]: Becoming Leader. State: Replica: 22b8a1f2db1942038acb0742e612d182, State: Running, Role: LEADER
I20260807 18:11:08.652822 29066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } }
I20260807 18:11:08.653434 29061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:08.656082 29067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22b8a1f2db1942038acb0742e612d182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } } }
I20260807 18:11:08.656157 29068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22b8a1f2db1942038acb0742e612d182. Latest consensus state: current_term: 1 leader_uuid: "22b8a1f2db1942038acb0742e612d182" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b8a1f2db1942038acb0742e612d182" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37295 } } }
I20260807 18:11:08.656495 29067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:08.656600 29068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:08.658165 29074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:08.663605 29074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:08.673533 29074 catalog_manager.cc:1347] Generated new cluster ID: f4ca6cdf2a0f43c8b95f938b6bfeeb6e
I20260807 18:11:08.673679 29074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:08.697423 29074 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:08.698905 29074 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:08.710825 29074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: Generated new TSK 0
I20260807 18:11:08.711817 29074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:08.724495 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.129:0
--local_ip_for_outbound_sockets=127.8.62.129
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--builtin_ntp_servers=127.8.62.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 18:11:09.078218 29086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:09.078510 29086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:09.078599 29086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:09.086066 29086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:09.086169 29086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:09.086277 29086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.129
I20260807 18:11:09.095793 29086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.62.129
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.129
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:09.097075 29086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:09.098875 29086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:09.112123 29091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:09.112176 29092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:09.112188 29094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:09.113425 29086 server_base.cc:1034] running on GCE node
I20260807 18:11:09.114005 29086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:09.114898 29086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:09.116196 29086 hybrid_clock.cc:648] HybridClock initialized: now 1786126269116118 us; error 45 us; skew 500 ppm
I20260807 18:11:09.116662 29086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:09.119295 29086 webserver.cc:460] Webserver started at http://127.8.62.129:37309/ using document root <none> and password file <none>
I20260807 18:11:09.120082 29086 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:09.120231 29086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:09.120721 29086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:09.123957 29086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data/instance:
uuid: "e4ed7c6aa68049b38d2db44114d5b9e3"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "b9b58d696e0d5d5f3e954beb42193af4"
server_key_iv: "5824aed01173dc131380ca871441fdb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.124950 29086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance:
uuid: "e4ed7c6aa68049b38d2db44114d5b9e3"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "b9b58d696e0d5d5f3e954beb42193af4"
server_key_iv: "5824aed01173dc131380ca871441fdb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.131266 29086 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:11:09.135695 29100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.137380 29086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:09.137591 29086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal
uuid: "e4ed7c6aa68049b38d2db44114d5b9e3"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "b9b58d696e0d5d5f3e954beb42193af4"
server_key_iv: "5824aed01173dc131380ca871441fdb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.137843 29086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:09.163121 29086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:09.164085 29086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:09.164440 29086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:09.165591 29086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:09.167371 29086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:09.167514 29086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.167658 29086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:09.167745 29086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.205478 29086 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.129:35701
I20260807 18:11:09.205582 29213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.129:35701 every 8 connection(s)
I20260807 18:11:09.206920 29086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/data/info.pb
I20260807 18:11:09.213758 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29086
I20260807 18:11:09.213958 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal/instance
I20260807 18:11:09.214394 8442 external_mini_cluster.cc:1428] Setting key 939fa7434427777514bf61c1683310de
I20260807 18:11:09.217430 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.130:0
--local_ip_for_outbound_sockets=127.8.62.130
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--builtin_ntp_servers=127.8.62.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:11:09.221432 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:09.209815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:47965 (local address 127.8.62.190:37295)
0807 18:11:09.210239 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:09.210244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:09.211595 (+ 1351us) server_negotiation.cc:407] Connection header received
0807 18:11:09.213737 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:09.213743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:09.213865 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:09.214011 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:09.215242 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.216305 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.218255 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.218769 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.218914 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:09.220453 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:09.220465 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:09.220468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:09.220489 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:09.220538 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:09.220601 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:09.220694 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:09.220945 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 18:11:09.221050 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":201,"threads_started":1}
I20260807 18:11:09.222545 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:09.210359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37295 (local address 127.8.62.129:47965)
0807 18:11:09.211412 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:09.211464 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:09.213418 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:09.214272 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:09.214281 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:09.214377 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:09.215057 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:09.215076 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.216512 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.216518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:09.218088 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:09.218102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.219045 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.219056 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:09.219321 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:09.220088 (+ 767us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:09.220092 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:09.220136 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:09.220229 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:09.220696 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:09.220712 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:09.220724 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 18:11:09.220935 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 18:11:09.221390 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":273,"threads_started":1}
I20260807 18:11:09.224189 29214 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37295
I20260807 18:11:09.224524 29214 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:09.225425 29214 heartbeater.cc:511] Master 127.8.62.190:37295 requested a full tablet report, sending...
I20260807 18:11:09.227708 29025 ts_manager.cc:194] Registered new tserver with Master: e4ed7c6aa68049b38d2db44114d5b9e3 (127.8.62.129:35701)
I20260807 18:11:09.229386 29025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.129:47965
W20260807 18:11:09.561048 29218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:09.561425 29218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:09.561589 29218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:09.569458 29218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:09.569615 29218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:09.569797 29218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.130
I20260807 18:11:09.579620 29218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.62.130
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.130
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:09.581463 29218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:09.583762 29218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:09.597790 29224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:09.597877 29226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:09.597865 29223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:09.599082 29218 server_base.cc:1034] running on GCE node
I20260807 18:11:09.599730 29218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:09.600829 29218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:09.602097 29218 hybrid_clock.cc:648] HybridClock initialized: now 1786126269602046 us; error 61 us; skew 500 ppm
I20260807 18:11:09.602455 29218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:09.604848 29218 webserver.cc:460] Webserver started at http://127.8.62.130:40419/ using document root <none> and password file <none>
I20260807 18:11:09.605562 29218 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:09.605696 29218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:09.606046 29218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:09.609133 29218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data/instance:
uuid: "c12b4647d19f467e9191455cbf6ea798"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "53b37e5822120f373444dea8c8ed40f7"
server_key_iv: "ef36d09468719f300b77f30ad2bb0af6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.609958 29218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance:
uuid: "c12b4647d19f467e9191455cbf6ea798"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "53b37e5822120f373444dea8c8ed40f7"
server_key_iv: "ef36d09468719f300b77f30ad2bb0af6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.615456 29218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:09.619138 29232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.620612 29218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 18:11:09.620791 29218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal
uuid: "c12b4647d19f467e9191455cbf6ea798"
format_stamp: "Formatted at 2026-08-07 18:11:09 on dist-test-slave-rnrw"
server_key: "53b37e5822120f373444dea8c8ed40f7"
server_key_iv: "ef36d09468719f300b77f30ad2bb0af6"
server_key_version: "encryptionkey@0"
I20260807 18:11:09.620992 29218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:09.634502 29218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:09.635421 29218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:09.635731 29218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:09.636780 29218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:09.638283 29218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:09.638370 29218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.638499 29218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:09.638552 29218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:09.674206 29218 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.130:43821
I20260807 18:11:09.674296 29347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.130:43821 every 8 connection(s)
I20260807 18:11:09.675603 29218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/data/info.pb
I20260807 18:11:09.675930 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29218
I20260807 18:11:09.676110 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal/instance
I20260807 18:11:09.676756 8442 external_mini_cluster.cc:1428] Setting key 799954720838251d1e6ef482e2c76add
I20260807 18:11:09.680213 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.62.131:0
--local_ip_for_outbound_sockets=127.8.62.131
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--builtin_ntp_servers=127.8.62.148:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 18:11:09.688993 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:09.678380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.130:40611 (local address 127.8.62.190:37295)
0807 18:11:09.678556 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:09.678560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:09.680286 (+ 1726us) server_negotiation.cc:407] Connection header received
0807 18:11:09.682207 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:09.682212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:09.682304 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:09.682414 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:09.683536 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.684270 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.685878 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.686287 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.686424 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:09.688105 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:09.688110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:09.688114 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:09.688134 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:09.688171 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:09.688183 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:09.688289 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:09.688563 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 18:11:09.688673 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:11:09.690531 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:09.679094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37295 (local address 127.8.62.130:40611)
0807 18:11:09.680132 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:09.680169 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:09.681911 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:09.682661 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:09.682669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:09.682765 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:09.683345 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:09.683362 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.684422 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.684428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:09.685725 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:09.685738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:09.686535 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:09.686543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:09.686795 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:09.687703 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:09.687705 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:09.687757 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:09.687847 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:09.688290 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:09.688304 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:09.688315 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 18:11:09.688524 (+ 209us) client_negotiation.cc:240] Negotiation successful
0807 18:11:09.688984 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":356,"threads_started":1}
I20260807 18:11:09.692088 29348 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37295
I20260807 18:11:09.692427 29348 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:09.693205 29348 heartbeater.cc:511] Master 127.8.62.190:37295 requested a full tablet report, sending...
I20260807 18:11:09.694669 29025 ts_manager.cc:194] Registered new tserver with Master: c12b4647d19f467e9191455cbf6ea798 (127.8.62.130:43821)
I20260807 18:11:09.695500 29025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.130:40611
W20260807 18:11:10.022663 29351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:10.022967 29351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:10.023120 29351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:10.030804 29351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:10.030918 29351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:10.031106 29351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.62.131
I20260807 18:11:10.040701 29351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.62.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.62.131
--webserver_port=0
--tserver_master_addrs=127.8.62.190:37295
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.62.131
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:10.042037 29351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:10.044019 29351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:10.057447 29359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:10.057410 29357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:10.057365 29356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:10.063812 29351 server_base.cc:1034] running on GCE node
I20260807 18:11:10.064527 29351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:10.065555 29351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:10.066833 29351 hybrid_clock.cc:648] HybridClock initialized: now 1786126270066773 us; error 43 us; skew 500 ppm
I20260807 18:11:10.067243 29351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 18:11:10.069733 29351 webserver.cc:460] Webserver started at http://127.8.62.131:43603/ using document root <none> and password file <none>
I20260807 18:11:10.070454 29351 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:10.070595 29351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:10.070935 29351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:10.073884 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data/instance:
uuid: "05368172700841e4a3d148fc2313ac43"
format_stamp: "Formatted at 2026-08-07 18:11:10 on dist-test-slave-rnrw"
server_key: "1665c42099c9246102da1a71d657d84b"
server_key_iv: "3c57e3cbdf618cd9c5fe771d29569ab2"
server_key_version: "encryptionkey@0"
I20260807 18:11:10.074687 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance:
uuid: "05368172700841e4a3d148fc2313ac43"
format_stamp: "Formatted at 2026-08-07 18:11:10 on dist-test-slave-rnrw"
server_key: "1665c42099c9246102da1a71d657d84b"
server_key_iv: "3c57e3cbdf618cd9c5fe771d29569ab2"
server_key_version: "encryptionkey@0"
I20260807 18:11:10.079866 29351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:10.083431 29365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:10.084813 29351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:10.084988 29351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal
uuid: "05368172700841e4a3d148fc2313ac43"
format_stamp: "Formatted at 2026-08-07 18:11:10 on dist-test-slave-rnrw"
server_key: "1665c42099c9246102da1a71d657d84b"
server_key_iv: "3c57e3cbdf618cd9c5fe771d29569ab2"
server_key_version: "encryptionkey@0"
I20260807 18:11:10.085222 29351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:10.110059 29351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:10.110950 29351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:10.111281 29351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:10.112300 29351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 18:11:10.113823 29351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 18:11:10.113950 29351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:10.114094 29351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 18:11:10.114184 29351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:10.150888 29351 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.131:45343
I20260807 18:11:10.150986 29479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.131:45343 every 8 connection(s)
I20260807 18:11:10.152290 29351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/data/info.pb
I20260807 18:11:10.159184 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29351
I20260807 18:11:10.159391 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal/instance
I20260807 18:11:10.159983 8442 external_mini_cluster.cc:1428] Setting key 3c4fee0ab3e30e4b28f0305bfc7df261
I20260807 18:11:10.166684 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.155335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.131:46897 (local address 127.8.62.190:37295)
0807 18:11:10.155572 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.155578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.157187 (+ 1609us) server_negotiation.cc:407] Connection header received
0807 18:11:10.159238 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.159244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.159349 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.159493 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:10.161045 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.162148 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.164028 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.164444 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.164601 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.165963 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:10.165968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:10.165971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:10.165991 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:10.166022 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:10.166034 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:10.166105 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.166344 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.166427 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:10.167940 29482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.155939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:37295 (local address 127.8.62.131:46897)
0807 18:11:10.157013 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.157053 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.158878 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.159806 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.159815 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.159934 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:10.160842 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.160866 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.162321 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.162326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.163887 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.163901 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.164729 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.164734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.164945 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.165642 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 18:11:10.165644 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:10.165686 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 18:11:10.165778 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:10.166120 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:10.166127 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:10.166133 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.166273 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.166641 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":324,"threads_started":1}
I20260807 18:11:10.169603 29480 heartbeater.cc:347] Connected to a master server at 127.8.62.190:37295
I20260807 18:11:10.169957 29480 heartbeater.cc:464] Registering TS with master...
I20260807 18:11:10.170773 29480 heartbeater.cc:511] Master 127.8.62.190:37295 requested a full tablet report, sending...
I20260807 18:11:10.172480 29025 ts_manager.cc:194] Registered new tserver with Master: 05368172700841e4a3d148fc2313ac43 (127.8.62.131:45343)
I20260807 18:11:10.173548 29025 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.62.131:46897
I20260807 18:11:10.175845 8442 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 18:11:10.187273 29216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.181026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49314 (local address 127.8.62.190:37295)
0807 18:11:10.181288 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.181296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.181629 (+ 333us) server_negotiation.cc:407] Connection header received
0807 18:11:10.181861 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.181867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.181961 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.182078 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:10.182624 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.183350 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.184827 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.185263 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.185405 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.186503 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:10.186510 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:10.186514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:10.186552 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:10.186586 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:10.186597 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:10.186697 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.186844 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.186943 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 18:11:10.191686 29025 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 18:11:10.198719 29025 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 18:11:10.223148 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.212919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42576 (local address 127.8.62.130:43821)
0807 18:11:10.215201 (+ 2282us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.215207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.215229 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:11:10.215301 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.215306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.215495 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.215689 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:10.216493 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.217942 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.220717 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.221966 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.222105 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.222344 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.222628 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.222801 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2132,"thread_start_us":2011,"threads_started":1}
I20260807 18:11:10.223229 29497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.213168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:43821 (local address 127.0.0.1:42576)
0807 18:11:10.214485 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.214507 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.214652 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.215916 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.215922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.215950 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:10.216302 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.216315 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.218086 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.218091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.220475 (+ 2384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.220488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.222245 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.222254 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.222423 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.222447 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.222566 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.222773 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"mutex_wait_us":49,"thread_start_us":1018,"threads_started":1}
I20260807 18:11:10.223256 29492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.211439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:45343 (local address 127.0.0.1:39882)
0807 18:11:10.212352 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.212400 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.212640 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.215028 (+ 2388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.215050 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.215085 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:10.215558 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.215574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.217615 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.217620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.220248 (+ 2628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.220263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.222091 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.222098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.222342 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.222402 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.222553 (+ 151us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.222778 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":222,"thread_start_us":360,"threads_started":1}
I20260807 18:11:10.223171 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.211625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.8.62.131:45343)
0807 18:11:10.214507 (+ 2882us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.214515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.214549 (+ 34us) server_negotiation.cc:407] Connection header received
0807 18:11:10.214629 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.214636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.214799 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.215491 (+ 692us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:10.215795 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.217470 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.220452 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.221870 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.222010 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.222286 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.222596 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.222773 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1540,"thread_start_us":1357,"threads_started":1}
I20260807 18:11:10.223992 29494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.212011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.129:35701 (local address 127.0.0.1:39410)
0807 18:11:10.214118 (+ 2107us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.214137 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.214269 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.215254 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.215258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.215278 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:10.215697 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.215710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.217750 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.217756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.219768 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.219784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.221886 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.221894 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.222100 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.223436 (+ 1336us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.223524 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.223671 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1968,"mutex_wait_us":613,"thread_start_us":1835,"threads_started":1}
I20260807 18:11:10.224354 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.212518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.8.62.129:35701)
0807 18:11:10.213701 (+ 1183us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.213713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.214417 (+ 704us) server_negotiation.cc:407] Connection header received
0807 18:11:10.214519 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.214528 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.214788 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.214985 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:10.216020 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.217605 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.219974 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.221321 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.221459 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.221760 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.223835 (+ 2075us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.223997 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":861,"threads_started":1}
I20260807 18:11:10.226884 29281 tablet_service.cc:1467] Processing CreateTablet for tablet 05281341c01a46e3a6d8991c918fbcde (DEFAULT_TABLE table=test-table [id=c87f261d644844ea90a746dfcd5c1260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:10.226881 29415 tablet_service.cc:1467] Processing CreateTablet for tablet 05281341c01a46e3a6d8991c918fbcde (DEFAULT_TABLE table=test-table [id=c87f261d644844ea90a746dfcd5c1260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:10.228283 29415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05281341c01a46e3a6d8991c918fbcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:10.228281 29281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05281341c01a46e3a6d8991c918fbcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:10.231024 29148 tablet_service.cc:1467] Processing CreateTablet for tablet 05281341c01a46e3a6d8991c918fbcde (DEFAULT_TABLE table=test-table [id=c87f261d644844ea90a746dfcd5c1260]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 18:11:10.232385 29148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05281341c01a46e3a6d8991c918fbcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:10.232489 29214 heartbeater.cc:503] Master 127.8.62.190:37295 was elected leader, sending a full tablet report...
I20260807 18:11:10.239627 29498 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Bootstrap starting.
I20260807 18:11:10.242813 29499 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Bootstrap starting.
I20260807 18:11:10.244860 29498 tablet_bootstrap.cc:654] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:10.246182 29500 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Bootstrap starting.
I20260807 18:11:10.246287 29498 log.cc:824] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:10.246481 29499 tablet_bootstrap.cc:654] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:10.248137 29499 log.cc:824] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:10.249970 29498 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: No bootstrap required, opened a new log
I20260807 18:11:10.250123 29500 tablet_bootstrap.cc:654] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:10.250290 29498 ts_tablet_manager.cc:1397] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260807 18:11:10.250844 29499 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: No bootstrap required, opened a new log
I20260807 18:11:10.251107 29499 ts_tablet_manager.cc:1397] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260807 18:11:10.252171 29500 log.cc:824] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:10.255394 29500 tablet_bootstrap.cc:492] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: No bootstrap required, opened a new log
I20260807 18:11:10.255687 29500 ts_tablet_manager.cc:1397] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260807 18:11:10.259639 29499 raft_consensus.cc:348] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.260177 29499 raft_consensus.cc:374] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:10.260284 29499 raft_consensus.cc:729] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12b4647d19f467e9191455cbf6ea798, State: Initialized, Role: FOLLOWER
I20260807 18:11:10.260975 29499 consensus_queue.cc:260] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.260949 29498 raft_consensus.cc:348] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.261402 29498 raft_consensus.cc:374] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:10.261492 29498 raft_consensus.cc:729] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05368172700841e4a3d148fc2313ac43, State: Initialized, Role: FOLLOWER
I20260807 18:11:10.262306 29348 heartbeater.cc:503] Master 127.8.62.190:37295 was elected leader, sending a full tablet report...
I20260807 18:11:10.262140 29498 consensus_queue.cc:260] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.263022 29499 ts_tablet_manager.cc:1428] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260807 18:11:10.264971 29480 heartbeater.cc:503] Master 127.8.62.190:37295 was elected leader, sending a full tablet report...
I20260807 18:11:10.265097 29498 ts_tablet_manager.cc:1428] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Time spent starting tablet: real 0.015s user 0.002s sys 0.014s
I20260807 18:11:10.266774 29500 raft_consensus.cc:348] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.267307 29500 raft_consensus.cc:374] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:10.267410 29500 raft_consensus.cc:729] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4ed7c6aa68049b38d2db44114d5b9e3, State: Initialized, Role: FOLLOWER
I20260807 18:11:10.268033 29500 consensus_queue.cc:260] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.269940 29500 ts_tablet_manager.cc:1428] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
W20260807 18:11:10.404335 29481 tablet.cc:2369] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:10.427990 29349 tablet.cc:2369] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 18:11:10.459612 29215 tablet.cc:2369] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:10.623540 29506 raft_consensus.cc:482] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 18:11:10.623823 29506 raft_consensus.cc:504] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.625597 29506 leader_election.cc:288] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05368172700841e4a3d148fc2313ac43 (127.8.62.131:45343), c12b4647d19f467e9191455cbf6ea798 (127.8.62.130:43821)
I20260807 18:11:10.633881 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.626066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:49625 (local address 127.8.62.131:45343)
0807 18:11:10.626325 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.626333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.626732 (+ 399us) server_negotiation.cc:407] Connection header received
0807 18:11:10.626954 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.626960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.627065 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.627160 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:10.627972 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.629139 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.631908 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.633065 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.633234 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.633309 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.633546 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.633679 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 18:11:10.634047 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.625893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.131:45343 (local address 127.8.62.129:49625)
0807 18:11:10.626610 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.626632 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.626807 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.627294 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.627297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.627318 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:10.627781 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.627797 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.629307 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.629311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.631699 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.631720 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.633254 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.633258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.633374 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.633399 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.633476 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.633644 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":59,"thread_start_us":365,"threads_started":1}
I20260807 18:11:10.634402 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.626865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.62.129:50413 (local address 127.8.62.130:43821)
0807 18:11:10.627083 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.627091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.627553 (+ 462us) server_negotiation.cc:407] Connection header received
0807 18:11:10.627815 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.627821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.627935 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.628014 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 18:11:10.628944 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.629768 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.632208 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.633417 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.633588 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.633652 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.634012 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.634143 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:11:10.634449 29508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.626655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.130:43821 (local address 127.8.62.129:50413)
0807 18:11:10.627416 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:10.627445 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:10.627605 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:10.628251 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:10.628258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:10.628289 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 18:11:10.628778 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.628789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.629920 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.629925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.632028 (+ 2103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:10.632038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.633623 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.633628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:10.633833 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.633863 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 18:11:10.633955 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 18:11:10.634108 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":52,"thread_start_us":437,"threads_started":1}
I20260807 18:11:10.635310 29435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05281341c01a46e3a6d8991c918fbcde" candidate_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05368172700841e4a3d148fc2313ac43" is_pre_election: true
I20260807 18:11:10.635474 29302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05281341c01a46e3a6d8991c918fbcde" candidate_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12b4647d19f467e9191455cbf6ea798" is_pre_election: true
I20260807 18:11:10.635957 29435 raft_consensus.cc:2454] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4ed7c6aa68049b38d2db44114d5b9e3 in term 0.
I20260807 18:11:10.635996 29302 raft_consensus.cc:2454] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4ed7c6aa68049b38d2db44114d5b9e3 in term 0.
I20260807 18:11:10.636754 29104 leader_election.cc:302] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05368172700841e4a3d148fc2313ac43, e4ed7c6aa68049b38d2db44114d5b9e3; no voters:
I20260807 18:11:10.637135 29506 raft_consensus.cc:2781] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 18:11:10.637291 29506 raft_consensus.cc:482] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 18:11:10.637401 29506 raft_consensus.cc:3037] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:10.639599 29506 raft_consensus.cc:504] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.640226 29506 leader_election.cc:288] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [CANDIDATE]: Term 1 election: Requested vote from peers 05368172700841e4a3d148fc2313ac43 (127.8.62.131:45343), c12b4647d19f467e9191455cbf6ea798 (127.8.62.130:43821)
I20260807 18:11:10.640741 29435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05281341c01a46e3a6d8991c918fbcde" candidate_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05368172700841e4a3d148fc2313ac43"
I20260807 18:11:10.640940 29435 raft_consensus.cc:3037] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:10.640920 29302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05281341c01a46e3a6d8991c918fbcde" candidate_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c12b4647d19f467e9191455cbf6ea798"
I20260807 18:11:10.641179 29302 raft_consensus.cc:3037] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:10.643492 29435 raft_consensus.cc:2454] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4ed7c6aa68049b38d2db44114d5b9e3 in term 1.
I20260807 18:11:10.643769 29302 raft_consensus.cc:2454] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4ed7c6aa68049b38d2db44114d5b9e3 in term 1.
I20260807 18:11:10.643987 29104 leader_election.cc:302] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05368172700841e4a3d148fc2313ac43, e4ed7c6aa68049b38d2db44114d5b9e3; no voters:
I20260807 18:11:10.644342 29506 raft_consensus.cc:2781] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:10.644826 29506 raft_consensus.cc:686] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 1 LEADER]: Becoming Leader. State: Replica: e4ed7c6aa68049b38d2db44114d5b9e3, State: Running, Role: LEADER
I20260807 18:11:10.645323 29506 consensus_queue.cc:237] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } }
I20260807 18:11:10.649916 29025 catalog_manager.cc:5495] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4ed7c6aa68049b38d2db44114d5b9e3 (127.8.62.129). New cstate: current_term: 1 leader_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4ed7c6aa68049b38d2db44114d5b9e3" member_type: VOTER last_known_addr { host: "127.8.62.129" port: 35701 } health_report { overall_health: HEALTHY } } }
I20260807 18:11:10.687098 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.681086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.8.62.129:35701)
0807 18:11:10.681277 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.681281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.681300 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:11:10.681463 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.681467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.681602 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.681728 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:10.682165 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.682892 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.684899 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.685340 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.685437 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.686021 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:10.686510 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:10.686641 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.686763 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.686831 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:11:10.696321 29302 raft_consensus.cc:1261] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 1 FOLLOWER]: Refusing update from remote peer e4ed7c6aa68049b38d2db44114d5b9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:10.696405 29435 raft_consensus.cc:1261] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 1 FOLLOWER]: Refusing update from remote peer e4ed7c6aa68049b38d2db44114d5b9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:10.697697 29509 consensus_queue.cc:1035] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c12b4647d19f467e9191455cbf6ea798" member_type: VOTER last_known_addr { host: "127.8.62.130" port: 43821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:10.698081 29506 consensus_queue.cc:1035] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05368172700841e4a3d148fc2313ac43" member_type: VOTER last_known_addr { host: "127.8.62.131" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:10.714789 29515 mvcc.cc:205] Tried to move back new op lower bound from 7315973204761415680 to 7315973204567134208. Current Snapshot: MvccSnapshot[applied={T|T < 7315973204761415680}]
I20260807 18:11:10.726809 29025 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:10.727129 29025 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 18:11:10.730870 29025 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22b8a1f2db1942038acb0742e612d182: Sending DeleteTablet for 3 replicas of tablet 05281341c01a46e3a6d8991c918fbcde
I20260807 18:11:10.732314 29148 tablet_service.cc:1514] Processing DeleteTablet for tablet 05281341c01a46e3a6d8991c918fbcde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:10 UTC) from {username='slave'} at 127.0.0.1:39410
I20260807 18:11:10.732443 29281 tablet_service.cc:1514] Processing DeleteTablet for tablet 05281341c01a46e3a6d8991c918fbcde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:10 UTC) from {username='slave'} at 127.0.0.1:42576
I20260807 18:11:10.732455 29415 tablet_service.cc:1514] Processing DeleteTablet for tablet 05281341c01a46e3a6d8991c918fbcde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 18:11:10 UTC) from {username='slave'} at 127.0.0.1:39882
I20260807 18:11:10.733502 29523 tablet_replica.cc:330] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: stopping tablet replica
I20260807 18:11:10.734216 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29086
I20260807 18:11:10.734184 29523 raft_consensus.cc:2229] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 18:11:10.735221 29523 raft_consensus.cc:2258] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:10.738233 29523 ts_tablet_manager.cc:1905] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:10.749382 29523 ts_tablet_manager.cc:1918] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:10.750133 29523 log.cc:1196] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-0/wal/wals/05281341c01a46e3a6d8991c918fbcde
I20260807 18:11:10.751149 29523 ts_tablet_manager.cc:1939] T 05281341c01a46e3a6d8991c918fbcde P e4ed7c6aa68049b38d2db44114d5b9e3: Deleting consensus metadata
I20260807 18:11:10.754015 29010 catalog_manager.cc:4841] TS e4ed7c6aa68049b38d2db44114d5b9e3 (127.8.62.129:35701): tablet 05281341c01a46e3a6d8991c918fbcde (table test-table [id=c87f261d644844ea90a746dfcd5c1260]) successfully deleted
I20260807 18:11:10.756273 29524 tablet_replica.cc:330] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: stopping tablet replica
I20260807 18:11:10.757112 29524 raft_consensus.cc:2229] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:10.757910 29524 raft_consensus.cc:2258] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:10.758991 29525 tablet_replica.cc:330] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: stopping tablet replica
I20260807 18:11:10.759565 29525 raft_consensus.cc:2229] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 18:11:10.760138 29525 raft_consensus.cc:2258] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 18:11:10.760610 29524 ts_tablet_manager.cc:1905] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:10.762792 29525 ts_tablet_manager.cc:1905] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 18:11:10.768364 29524 ts_tablet_manager.cc:1918] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:10.768574 29524 log.cc:1196] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-1/wal/wals/05281341c01a46e3a6d8991c918fbcde
I20260807 18:11:10.769140 29524 ts_tablet_manager.cc:1939] T 05281341c01a46e3a6d8991c918fbcde P c12b4647d19f467e9191455cbf6ea798: Deleting consensus metadata
I20260807 18:11:10.770525 29525 ts_tablet_manager.cc:1918] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 18:11:10.770709 29525 log.cc:1196] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Deleting WAL directory at /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786125932226923-8442-0/minicluster-data/ts-2/wal/wals/05281341c01a46e3a6d8991c918fbcde
I20260807 18:11:10.771267 29525 ts_tablet_manager.cc:1939] T 05281341c01a46e3a6d8991c918fbcde P 05368172700841e4a3d148fc2313ac43: Deleting consensus metadata
I20260807 18:11:10.771260 29012 catalog_manager.cc:4841] TS c12b4647d19f467e9191455cbf6ea798 (127.8.62.130:43821): tablet 05281341c01a46e3a6d8991c918fbcde (table test-table [id=c87f261d644844ea90a746dfcd5c1260]) successfully deleted
I20260807 18:11:10.771726 29495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.735886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.8.62.129:35701)
0807 18:11:10.736088 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.736095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.755476 (+ 19381us) server_negotiation.cc:407] Connection header received
0807 18:11:10.755753 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.755760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.755873 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.756013 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:10.756829 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.758236 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.760484 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.761317 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.761434 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.768747 (+ 7313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:10.768763 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:10.768784 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:10.769745 (+ 961us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:10.769820 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:10.769853 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:10.769954 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.770877 (+ 923us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.771218 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:11:10.772733 29209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:10.773171 29012 catalog_manager.cc:4841] TS 05368172700841e4a3d148fc2313ac43 (127.8.62.131:45343): tablet 05281341c01a46e3a6d8991c918fbcde (table test-table [id=c87f261d644844ea90a746dfcd5c1260]) successfully deleted
I20260807 18:11:10.881669 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29086
I20260807 18:11:10.902803 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29218
I20260807 18:11:10.910511 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:10.903793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42590 (local address 127.8.62.130:43821)
0807 18:11:10.904000 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:10.904005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:10.904026 (+ 21us) server_negotiation.cc:407] Connection header received
0807 18:11:10.904132 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:10.904136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:10.904224 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:10.904330 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:10.904814 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.905706 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.907132 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:10.907624 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:10.907732 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:10.909027 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:10.909043 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:10.909059 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:10.909701 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:10.909754 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:10.909775 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:10.909881 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:10.910109 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 18:11:10.910213 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:11:10.911342 29343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:11.000895 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29218
I20260807 18:11:11.022387 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29351
I20260807 18:11:11.029579 29493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:11.023254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39890 (local address 127.8.62.131:45343)
0807 18:11:11.023466 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:11.023472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:11.023505 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:11:11.023603 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:11.023608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:11.023713 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:11.023842 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:11.024259 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:11.025113 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:11.026388 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:11.026818 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:11.026956 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:11.028089 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:11.028099 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:11.028115 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:11.028781 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:11.028832 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:11.028853 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:11.028986 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:11.029123 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 18:11:11.029235 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 18:11:11.030263 29475 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:11.119831 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29351
I20260807 18:11:11.138403 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 28994
I20260807 18:11:11.139660 29056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:11.255450 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 28994
2026-08-07T18:11:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3192 ms)
[----------] 6 tests from EncryptionPolicyTest (23628 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:11 dist-test-slave-rnrw krb5kdc[29533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:11 dist-test-slave-rnrw krb5kdc[29533](info): set up 2 sockets
Aug 07 18:11:11 dist-test-slave-rnrw krb5kdc[29533](info): commencing operation
krb5kdc: starting...
W20260807 18:11:12.411482 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.008s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:12 dist-test-slave-rnrw krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:12.692890 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44529
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:13.044934 29546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:13.045245 29546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:13.045338 29546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:13.052922 29546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:13.053050 29546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:13.053097 29546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:13.053160 29546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:13.053205 29546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:13.063304 29546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44529
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:13.064638 29546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:13.065886 29546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:13.078058 29554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:13.078020 29552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:13.078187 29551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:13.078912 29546 server_base.cc:1034] running on GCE node
I20260807 18:11:13.079879 29546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:13.081404 29546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:13.082630 29546 hybrid_clock.cc:648] HybridClock initialized: now 1786126273082580 us; error 34 us; skew 500 ppm
I20260807 18:11:13.083045 29546 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:13 dist-test-slave-rnrw krb5kdc[29533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126273, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:13.086539 29546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:13.088259 29546 webserver.cc:460] Webserver started at http://127.0.0.1:46531/ using document root <none> and password file <none>
I20260807 18:11:13.089058 29546 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:13.089186 29546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:13.089449 29546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:13.092722 29546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "20fc469b808a4724b0b3c21880f25b46"
format_stamp: "Formatted at 2026-08-07 18:11:13 on dist-test-slave-rnrw"
server_key: "d817fd529b2ec4c26cce0f3d01cae285"
server_key_iv: "21031486bff1662fdaaffc55dcae0d6a"
server_key_version: "encryptionkey@0"
I20260807 18:11:13.093458 29546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "20fc469b808a4724b0b3c21880f25b46"
format_stamp: "Formatted at 2026-08-07 18:11:13 on dist-test-slave-rnrw"
server_key: "d817fd529b2ec4c26cce0f3d01cae285"
server_key_iv: "21031486bff1662fdaaffc55dcae0d6a"
server_key_version: "encryptionkey@0"
I20260807 18:11:13.098861 29546 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:13.102708 29561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:13.104167 29546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:11:13.104398 29546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "20fc469b808a4724b0b3c21880f25b46"
format_stamp: "Formatted at 2026-08-07 18:11:13 on dist-test-slave-rnrw"
server_key: "d817fd529b2ec4c26cce0f3d01cae285"
server_key_iv: "21031486bff1662fdaaffc55dcae0d6a"
server_key_version: "encryptionkey@0"
I20260807 18:11:13.104650 29546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:13.117805 29546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:13.122627 29546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:13.122893 29546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:13.143168 29546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44529
I20260807 18:11:13.143200 29612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44529 every 8 connection(s)
I20260807 18:11:13.144742 29546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:13.149302 29613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:13.151257 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29546
I20260807 18:11:13.151508 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:13.152257 8442 external_mini_cluster.cc:1428] Setting key f23dd778b104eee846e425172be0c8af
I20260807 18:11:13.163985 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46: Bootstrap starting.
I20260807 18:11:13.168073 29613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46: Neither blocks nor log segments found. Creating new log.
Aug 07 18:11:13 dist-test-slave-rnrw krb5kdc[29533](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:11:13.169734 29613 log.cc:824] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:13.173691 29613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46: No bootstrap required, opened a new log
I20260807 18:11:13.179796 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:13.154564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53263 (local address 127.0.0.1:44529)
0807 18:11:13.155486 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:13.155503 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:13.155562 (+ 59us) server_negotiation.cc:407] Connection header received
0807 18:11:13.158094 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:13.158125 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:13.158685 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:13.159206 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:13.160775 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:13.162888 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:13.164379 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:13.165450 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:13.165657 (+ 207us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:13.170744 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:13.170761 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:13.170789 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:13.170844 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:13.174654 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:13.175882 (+ 1228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:13.175891 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:13.175898 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:13.176125 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:13.176695 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:13.176699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:13.176701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:13.177336 (+ 635us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:13.177611 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:13.178207 (+ 596us) server_negotiation.cc:299] Negotiation successful
0807 18:11:13.178606 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":386,"threads_started":1}
I20260807 18:11:13.183162 29613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } }
I20260807 18:11:13.183686 29613 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:13.183820 29613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20fc469b808a4724b0b3c21880f25b46, State: Initialized, Role: FOLLOWER
I20260807 18:11:13.184470 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:13.184497 29613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } }
I20260807 18:11:13.184871 29613 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:13.185030 29613 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:13.185221 29613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:13.187503 29613 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } }
I20260807 18:11:13.188213 29613 leader_election.cc:302] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20fc469b808a4724b0b3c21880f25b46; no voters:
I20260807 18:11:13.188894 29613 leader_election.cc:288] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:13.189051 29622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:13.189751 29622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [term 1 LEADER]: Becoming Leader. State: Replica: 20fc469b808a4724b0b3c21880f25b46, State: Running, Role: LEADER
I20260807 18:11:13.190471 29622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } }
I20260807 18:11:13.192737 29622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20fc469b808a4724b0b3c21880f25b46. Latest consensus state: current_term: 1 leader_uuid: "20fc469b808a4724b0b3c21880f25b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } } }
I20260807 18:11:13.193140 29622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:13.193636 29624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20fc469b808a4724b0b3c21880f25b46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20fc469b808a4724b0b3c21880f25b46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44529 } } }
I20260807 18:11:13.194178 29624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:13.197232 29613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:13.197752 29627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:13.203315 29627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:13.214156 29616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:13.190402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59881 (local address 127.0.0.1:44529)
0807 18:11:13.190597 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:13.190605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:13.190954 (+ 349us) server_negotiation.cc:407] Connection header received
0807 18:11:13.191298 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:13.191307 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:13.191414 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:13.191550 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:13.197199 (+ 5649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:13.198507 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:13.200493 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:13.201196 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:13.201338 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:13.205139 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:13.205149 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:13.205158 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:13.205200 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:13.209217 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:13.210347 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:13.210352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:13.210355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:13.210515 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:13.211100 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:13.211104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:13.211107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:13.211554 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:13.211856 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:13.213345 (+ 1489us) server_negotiation.cc:299] Negotiation successful
0807 18:11:13.213607 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 18:11:13.221038 29627 catalog_manager.cc:1347] Generated new cluster ID: c0cacda5f4434b098ecebf7d781a7387
I20260807 18:11:13.221186 29627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:13.236497 29627 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:13.238350 29627 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:13.249786 29627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20fc469b808a4724b0b3c21880f25b46: Generated new TSK 0
I20260807 18:11:13.250933 29627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:13.266551 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29546
I20260807 18:11:13.268091 29608 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:13.362596 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29546
W20260807 18:11:13.382284 29531 connection.cc:535] client connection to 127.0.0.1:44529 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44529): Connection reset by peer (error 104)
2026-08-07T18:11:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2101 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:13 dist-test-slave-rnrw krb5kdc[29645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:13 dist-test-slave-rnrw krb5kdc[29645](info): set up 2 sockets
Aug 07 18:11:13 dist-test-slave-rnrw krb5kdc[29645](info): commencing operation
krb5kdc: starting...
W20260807 18:11:14.521777 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:14 dist-test-slave-rnrw krb5kdc[29645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:14.820477 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43805
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:15.165602 29660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:15.165997 29660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:15.166170 29660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:15.173718 29660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:15.173889 29660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:15.173993 29660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:15.174101 29660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:15.174180 29660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:15.184489 29660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:15.185966 29660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:15.187361 29660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:15.200065 29665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:15.200156 29668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:15.200088 29666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:15.201135 29660 server_base.cc:1034] running on GCE node
I20260807 18:11:15.202005 29660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:15.203486 29660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:15.204686 29660 hybrid_clock.cc:648] HybridClock initialized: now 1786126275204649 us; error 27 us; skew 500 ppm
I20260807 18:11:15.205022 29660 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:15 dist-test-slave-rnrw krb5kdc[29645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126275, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:15.208500 29660 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:15.210146 29660 webserver.cc:460] Webserver started at http://127.0.0.1:44345/ using document root <none> and password file <none>
I20260807 18:11:15.210964 29660 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:15.211084 29660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:15.211388 29660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:15.214725 29660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "f36ae69307574f479e3c074d8442bf6c"
format_stamp: "Formatted at 2026-08-07 18:11:15 on dist-test-slave-rnrw"
server_key: "0d958372b3758be49278fefed146fe49"
server_key_iv: "14c6bd7ed7eb98f0ae090bb762e90655"
server_key_version: "encryptionkey@0"
I20260807 18:11:15.215510 29660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "f36ae69307574f479e3c074d8442bf6c"
format_stamp: "Formatted at 2026-08-07 18:11:15 on dist-test-slave-rnrw"
server_key: "0d958372b3758be49278fefed146fe49"
server_key_iv: "14c6bd7ed7eb98f0ae090bb762e90655"
server_key_version: "encryptionkey@0"
I20260807 18:11:15.221202 29660 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 18:11:15.224956 29675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:15.226363 29660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 18:11:15.226568 29660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "f36ae69307574f479e3c074d8442bf6c"
format_stamp: "Formatted at 2026-08-07 18:11:15 on dist-test-slave-rnrw"
server_key: "0d958372b3758be49278fefed146fe49"
server_key_iv: "14c6bd7ed7eb98f0ae090bb762e90655"
server_key_version: "encryptionkey@0"
I20260807 18:11:15.226758 29660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:15.243225 29660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:15.248039 29660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:15.248369 29660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:15.268240 29660 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43805
I20260807 18:11:15.268281 29726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43805 every 8 connection(s)
I20260807 18:11:15.269742 29660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:15.273802 29727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:15.279450 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29660
I20260807 18:11:15.279681 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:15.280175 8442 external_mini_cluster.cc:1428] Setting key 27bfa958995fa1ceb852d4d4fb6cd463
I20260807 18:11:15.285645 29727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c: Bootstrap starting.
I20260807 18:11:15.289170 29727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:15.290231 29727 log.cc:824] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:15.293744 29727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c: No bootstrap required, opened a new log
I20260807 18:11:15.294824 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:15.282134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50121 (local address 127.0.0.1:43805)
0807 18:11:15.283066 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:15.283085 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:15.283132 (+ 47us) server_negotiation.cc:407] Connection header received
0807 18:11:15.284941 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:15.284966 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:15.285617 (+ 651us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:15.286108 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:15.286782 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:15.288155 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:15.289627 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:15.290326 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:15.290502 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:15.291937 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:15.291948 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:15.291966 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:15.292716 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:15.292816 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:15.292853 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:15.293018 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:15.293319 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 18:11:15.293491 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":426,"threads_started":1}
I20260807 18:11:15.299500 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:15.304728 29727 raft_consensus.cc:348] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } }
I20260807 18:11:15.305244 29727 raft_consensus.cc:374] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:15.305563 29727 raft_consensus.cc:729] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36ae69307574f479e3c074d8442bf6c, State: Initialized, Role: FOLLOWER
I20260807 18:11:15.306377 29727 consensus_queue.cc:260] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } }
I20260807 18:11:15.306743 29727 raft_consensus.cc:388] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:15.306910 29727 raft_consensus.cc:482] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:15.307145 29727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:15.310585 29727 raft_consensus.cc:504] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } }
I20260807 18:11:15.311342 29727 leader_election.cc:302] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f36ae69307574f479e3c074d8442bf6c; no voters:
I20260807 18:11:15.312319 29727 leader_election.cc:288] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:15.312487 29737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:15.313862 29737 raft_consensus.cc:686] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [term 1 LEADER]: Becoming Leader. State: Replica: f36ae69307574f479e3c074d8442bf6c, State: Running, Role: LEADER
I20260807 18:11:15.314241 29730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:15.305257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37017 (local address 127.0.0.1:43805)
0807 18:11:15.305606 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:15.305616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:15.305953 (+ 337us) server_negotiation.cc:407] Connection header received
0807 18:11:15.306177 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:15.306182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:15.306254 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:15.306349 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:15.307112 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:15.308095 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:15.310069 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:15.310871 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:15.311008 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:15.312528 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:15.312535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:15.312562 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:15.312595 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:15.312679 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:15.312722 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:15.313531 (+ 809us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:15.313662 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 18:11:15.313748 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 18:11:15.314765 29737 consensus_queue.cc:237] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } }
I20260807 18:11:15.314991 29727 sys_catalog.cc:564] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:15.318498 29739 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f36ae69307574f479e3c074d8442bf6c. Latest consensus state: current_term: 1 leader_uuid: "f36ae69307574f479e3c074d8442bf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } } }
I20260807 18:11:15.318527 29738 sys_catalog.cc:455] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f36ae69307574f479e3c074d8442bf6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f36ae69307574f479e3c074d8442bf6c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43805 } } }
I20260807 18:11:15.319063 29738 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:15.319061 29739 sys_catalog.cc:458] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:15.321382 29745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:15.328413 29745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:15.339917 29745 catalog_manager.cc:1347] Generated new cluster ID: 625547317bb84c50ba5269105b2d455b
I20260807 18:11:15.340072 29745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:15.361351 29745 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:15.363343 29745 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:15.374787 29745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f36ae69307574f479e3c074d8442bf6c: Generated new TSK 0
I20260807 18:11:15.375931 29745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:15.389889 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29660
W20260807 18:11:15.391762 29722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50121
W20260807 18:11:15.392666 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29660: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:15.392868 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29660
W20260807 18:11:15.412005 29643 connection.cc:535] client connection to 127.0.0.1:43805 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43805): Connection reset by peer (error 104)
2026-08-07T18:11:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2029 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:15 dist-test-slave-rnrw krb5kdc[29758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:15 dist-test-slave-rnrw krb5kdc[29758](info): set up 2 sockets
Aug 07 18:11:15 dist-test-slave-rnrw krb5kdc[29758](info): commencing operation
krb5kdc: starting...
W20260807 18:11:16.543604 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:16 dist-test-slave-rnrw krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:16.836688 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:17.198616 29771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:17.198938 29771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:17.199146 29771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:17.206781 29771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:17.206916 29771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:17.207016 29771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:17.207111 29771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:17.207183 29771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:17.217787 29771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43811
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:17.219234 29771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:17.220708 29771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:17.233870 29777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:17.233819 29776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:17.233757 29779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:17.234863 29771 server_base.cc:1034] running on GCE node
I20260807 18:11:17.235827 29771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:17.237504 29771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:17.238765 29771 hybrid_clock.cc:648] HybridClock initialized: now 1786126277238720 us; error 46 us; skew 500 ppm
I20260807 18:11:17.239192 29771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:17 dist-test-slave-rnrw krb5kdc[29758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126277, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:17.246872 29771 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:17.248710 29771 webserver.cc:460] Webserver started at http://127.0.0.1:44437/ using document root <none> and password file <none>
I20260807 18:11:17.249476 29771 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:17.249595 29771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:17.249871 29771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:17.253197 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "caa14f7e91374bab8331189d3941afab"
format_stamp: "Formatted at 2026-08-07 18:11:17 on dist-test-slave-rnrw"
server_key: "50465ad89ea6e71a6a3ccd98c5dc3f8e"
server_key_iv: "67258b93f96b2da22271f62616caf8f1"
server_key_version: "encryptionkey@0"
I20260807 18:11:17.253981 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "caa14f7e91374bab8331189d3941afab"
format_stamp: "Formatted at 2026-08-07 18:11:17 on dist-test-slave-rnrw"
server_key: "50465ad89ea6e71a6a3ccd98c5dc3f8e"
server_key_iv: "67258b93f96b2da22271f62616caf8f1"
server_key_version: "encryptionkey@0"
I20260807 18:11:17.259757 29771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:17.263736 29786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:17.265324 29771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:17.265537 29771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "caa14f7e91374bab8331189d3941afab"
format_stamp: "Formatted at 2026-08-07 18:11:17 on dist-test-slave-rnrw"
server_key: "50465ad89ea6e71a6a3ccd98c5dc3f8e"
server_key_iv: "67258b93f96b2da22271f62616caf8f1"
server_key_version: "encryptionkey@0"
I20260807 18:11:17.265785 29771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:17.272096 29771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:17.276965 29771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:17.277243 29771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:17.297493 29771 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43811
I20260807 18:11:17.297528 29838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43811 every 8 connection(s)
I20260807 18:11:17.299053 29771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:17.302994 29839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:17.306953 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29771
I20260807 18:11:17.307199 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:17.307682 8442 external_mini_cluster.cc:1428] Setting key 7a6c70f2b48ccd304016e7b2eff615a4
I20260807 18:11:17.316191 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab: Bootstrap starting.
I20260807 18:11:17.318157 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:17.309775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49573 (local address 127.0.0.1:43811)
0807 18:11:17.310876 (+ 1101us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:17.310894 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:17.310948 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:11:17.314023 (+ 3075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:17.314057 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:17.314550 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:17.315045 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:17.316098 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:17.316106 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:17.316122 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:17.316141 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:17.316223 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:17.316257 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:17.316479 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:17.316647 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:11:17.316858 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":844,"thread_start_us":600,"threads_started":1}
I20260807 18:11:17.319973 29839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:17.321360 29839 log.cc:824] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:17.322242 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:17.324715 29839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab: No bootstrap required, opened a new log
I20260807 18:11:17.330199 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:17.327473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46511 (local address 127.0.0.1:43811)
0807 18:11:17.327679 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:17.327685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:17.328052 (+ 367us) server_negotiation.cc:407] Connection header received
0807 18:11:17.328207 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:17.328213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:17.328279 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:17.328397 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:17.329406 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:17.329413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:17.329417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:17.329436 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:17.329484 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:17.329503 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:17.329628 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:17.329796 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 18:11:17.329860 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:11:17.334061 29839 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } }
I20260807 18:11:17.334466 29839 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:17.334568 29839 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa14f7e91374bab8331189d3941afab, State: Initialized, Role: FOLLOWER
I20260807 18:11:17.335222 29839 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } }
I20260807 18:11:17.335470 29839 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:17.335621 29839 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:17.335827 29839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:17.338318 29839 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } }
I20260807 18:11:17.338876 29839 leader_election.cc:302] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa14f7e91374bab8331189d3941afab; no voters:
I20260807 18:11:17.339526 29839 leader_election.cc:288] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:17.339676 29849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:17.340360 29849 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [term 1 LEADER]: Becoming Leader. State: Replica: caa14f7e91374bab8331189d3941afab, State: Running, Role: LEADER
I20260807 18:11:17.340947 29849 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } }
I20260807 18:11:17.341691 29839 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:17.344180 29851 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa14f7e91374bab8331189d3941afab. Latest consensus state: current_term: 1 leader_uuid: "caa14f7e91374bab8331189d3941afab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } } }
I20260807 18:11:17.344177 29850 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa14f7e91374bab8331189d3941afab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa14f7e91374bab8331189d3941afab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43811 } } }
I20260807 18:11:17.344944 29851 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:17.345036 29850 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:17.347072 29856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:17.353122 29856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:17.363969 29856 catalog_manager.cc:1347] Generated new cluster ID: 3027898d73ae4326a947282e47d42cfd
I20260807 18:11:17.364118 29856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:17.378396 29856 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:17.378937 29856 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:17.393455 29856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa14f7e91374bab8331189d3941afab: Generated new TSK 0
I20260807 18:11:17.394397 29856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:17.399570 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29771
W20260807 18:11:17.400954 29834 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49573
W20260807 18:11:17.401823 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29771: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:17.402020 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29771
2026-08-07T18:11:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2009 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:17 dist-test-slave-rnrw krb5kdc[29870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:17 dist-test-slave-rnrw krb5kdc[29870](info): set up 2 sockets
Aug 07 18:11:17 dist-test-slave-rnrw krb5kdc[29870](info): commencing operation
krb5kdc: starting...
W20260807 18:11:18.541307 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:18 dist-test-slave-rnrw krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:18.840073 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:19.186540 29883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:19.186887 29883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:19.187062 29883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:19.194702 29883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:19.194837 29883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:19.194921 29883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:19.195041 29883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:19.195106 29883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:19.205374 29883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:19.206784 29883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:19.208112 29883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:19.220230 29888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:19.220284 29889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:19.220316 29891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:19.221165 29883 server_base.cc:1034] running on GCE node
I20260807 18:11:19.222046 29883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:19.223565 29883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:19.224809 29883 hybrid_clock.cc:648] HybridClock initialized: now 1786126279224768 us; error 48 us; skew 500 ppm
I20260807 18:11:19.225185 29883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:19 dist-test-slave-rnrw krb5kdc[29870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126279, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:19.228650 29883 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:19.230494 29883 webserver.cc:460] Webserver started at http://127.0.0.1:37823/ using document root <none> and password file <none>
I20260807 18:11:19.231283 29883 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:19.231446 29883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:19.231792 29883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:19.235165 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "198fcae723b04e8ba144ae4650a782ae"
format_stamp: "Formatted at 2026-08-07 18:11:19 on dist-test-slave-rnrw"
server_key: "7db9928f5767242b799e2501150a06af"
server_key_iv: "7a1c7519d13b3035c6938bf94a30d346"
server_key_version: "encryptionkey@0"
I20260807 18:11:19.236030 29883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "198fcae723b04e8ba144ae4650a782ae"
format_stamp: "Formatted at 2026-08-07 18:11:19 on dist-test-slave-rnrw"
server_key: "7db9928f5767242b799e2501150a06af"
server_key_iv: "7a1c7519d13b3035c6938bf94a30d346"
server_key_version: "encryptionkey@0"
I20260807 18:11:19.242027 29883 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 18:11:19.246217 29898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:19.247761 29883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 18:11:19.248011 29883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "198fcae723b04e8ba144ae4650a782ae"
format_stamp: "Formatted at 2026-08-07 18:11:19 on dist-test-slave-rnrw"
server_key: "7db9928f5767242b799e2501150a06af"
server_key_iv: "7a1c7519d13b3035c6938bf94a30d346"
server_key_version: "encryptionkey@0"
I20260807 18:11:19.248263 29883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:19.281883 29883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:19.286957 29883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:19.287302 29883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:19.307957 29883 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44071
I20260807 18:11:19.308019 29950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44071 every 8 connection(s)
I20260807 18:11:19.309515 29883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:19.310672 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29883
I20260807 18:11:19.310873 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:19.311406 8442 external_mini_cluster.cc:1428] Setting key 5793b8a57d4d0e0153b40f2b3f202c85
I20260807 18:11:19.314651 29951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:19.325749 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae: Bootstrap starting.
Aug 07 18:11:19 dist-test-slave-rnrw krb5kdc[29870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:11:19.329340 29951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:19.330570 29951 log.cc:824] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:19.333560 29951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae: No bootstrap required, opened a new log
I20260807 18:11:19.335264 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:19.313398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35701 (local address 127.0.0.1:44071)
0807 18:11:19.314264 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:19.314278 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:19.314324 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:11:19.316217 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:19.316240 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:19.316721 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:19.317170 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:19.318767 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:19.320258 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:19.321948 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:19.322736 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:19.322964 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:19.327812 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:19.327821 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:19.327836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:19.327884 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:19.331229 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:19.332002 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:19.332009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:19.332015 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:19.332175 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:19.332609 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:19.332613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:19.332615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:19.333118 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:19.333436 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:19.333874 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 18:11:19.334314 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":368,"threads_started":1}
I20260807 18:11:19.339437 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:19.342862 29951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } }
I20260807 18:11:19.343305 29951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:19.343415 29951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198fcae723b04e8ba144ae4650a782ae, State: Initialized, Role: FOLLOWER
I20260807 18:11:19.344177 29951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } }
I20260807 18:11:19.344504 29951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:19.344677 29951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:19.344882 29951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:19.347612 29951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } }
I20260807 18:11:19.348210 29951 leader_election.cc:302] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198fcae723b04e8ba144ae4650a782ae; no voters:
I20260807 18:11:19.348956 29951 leader_election.cc:288] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:19.349116 29961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:19.349818 29961 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [term 1 LEADER]: Becoming Leader. State: Replica: 198fcae723b04e8ba144ae4650a782ae, State: Running, Role: LEADER
I20260807 18:11:19.350531 29961 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } }
I20260807 18:11:19.353196 29961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198fcae723b04e8ba144ae4650a782ae. Latest consensus state: current_term: 1 leader_uuid: "198fcae723b04e8ba144ae4650a782ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } } }
I20260807 18:11:19.353608 29961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:19.353790 29951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:19.353822 29963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "198fcae723b04e8ba144ae4650a782ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198fcae723b04e8ba144ae4650a782ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44071 } } }
I20260807 18:11:19.354241 29963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:19.362360 29965 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:19.364377 29954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:19.344031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40087 (local address 127.0.0.1:44071)
0807 18:11:19.344239 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:19.344247 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:19.344779 (+ 532us) server_negotiation.cc:407] Connection header received
0807 18:11:19.345011 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:19.345018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:19.345107 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:19.345237 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:19.346664 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:19.347738 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:19.349900 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:19.350714 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:19.350843 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:19.354963 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:19.354976 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:19.354985 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:19.355043 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:19.360607 (+ 5564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:19.361717 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:19.361726 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:19.361731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:19.361962 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:19.362556 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:19.362564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:19.362571 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:19.362905 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:19.363247 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:19.363612 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 18:11:19.363864 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:11:19.369596 29965 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:19.380132 29965 catalog_manager.cc:1347] Generated new cluster ID: d405568709834a53959dc1a174d705b7
I20260807 18:11:19.380272 29965 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:19.407845 29965 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:19.410143 29965 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:19.428143 29965 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 198fcae723b04e8ba144ae4650a782ae: Generated new TSK 0
I20260807 18:11:19.429234 29965 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:19.445366 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29883
I20260807 18:11:19.446491 29946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:19.541734 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29883
2026-08-07T18:11:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2139 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:19 dist-test-slave-rnrw krb5kdc[29984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:19 dist-test-slave-rnrw krb5kdc[29984](info): set up 2 sockets
Aug 07 18:11:19 dist-test-slave-rnrw krb5kdc[29984](info): commencing operation
krb5kdc: starting...
W20260807 18:11:20.684067 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:20 dist-test-slave-rnrw krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:20.979964 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46457
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:21.336741 29998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:21.337049 29998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:21.337198 29998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:21.344671 29998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:21.344791 29998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:21.344870 29998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:21.344970 29998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:21.345227 29998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:21.355574 29998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:21.356945 29998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:21.358263 29998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:21.370384 30004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:21.370478 30003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:21.370464 30007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:21.371480 29998 server_base.cc:1034] running on GCE node
I20260807 18:11:21.372272 29998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:21.373773 29998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:21.375018 29998 hybrid_clock.cc:648] HybridClock initialized: now 1786126281374944 us; error 43 us; skew 500 ppm
I20260807 18:11:21.375411 29998 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:21 dist-test-slave-rnrw krb5kdc[29984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126281, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:21.378940 29998 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:21.380640 29998 webserver.cc:460] Webserver started at http://127.0.0.1:39459/ using document root <none> and password file <none>
I20260807 18:11:21.381438 29998 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:21.381588 29998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:21.381911 29998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:21.385116 29998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "c0487d7514794c4593f7f6fcd8503832"
format_stamp: "Formatted at 2026-08-07 18:11:21 on dist-test-slave-rnrw"
server_key: "a19699d07cb70af050fe2c313b195b92"
server_key_iv: "7fb8c0231572a1c9b3cd3072a79484ae"
server_key_version: "encryptionkey@0"
I20260807 18:11:21.385921 29998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "c0487d7514794c4593f7f6fcd8503832"
format_stamp: "Formatted at 2026-08-07 18:11:21 on dist-test-slave-rnrw"
server_key: "a19699d07cb70af050fe2c313b195b92"
server_key_iv: "7fb8c0231572a1c9b3cd3072a79484ae"
server_key_version: "encryptionkey@0"
I20260807 18:11:21.391587 29998 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 18:11:21.395268 30014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:21.396744 29998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:21.396934 29998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "c0487d7514794c4593f7f6fcd8503832"
format_stamp: "Formatted at 2026-08-07 18:11:21 on dist-test-slave-rnrw"
server_key: "a19699d07cb70af050fe2c313b195b92"
server_key_iv: "7fb8c0231572a1c9b3cd3072a79484ae"
server_key_version: "encryptionkey@0"
I20260807 18:11:21.397193 29998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:21.412896 29998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:21.417929 29998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:21.418263 29998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:21.438330 29998 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46457
I20260807 18:11:21.438385 30065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46457 every 8 connection(s)
I20260807 18:11:21.439957 29998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:21.443943 30066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:21.447908 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 29998
I20260807 18:11:21.448132 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:21.448649 8442 external_mini_cluster.cc:1428] Setting key 8bbcb3fa569d20da7ad4061b113371b8
I20260807 18:11:21.457017 30066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832: Bootstrap starting.
I20260807 18:11:21.460686 30066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:21.461977 30066 log.cc:824] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:21.462112 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:21.450420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40035 (local address 127.0.0.1:46457)
0807 18:11:21.451266 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:21.451278 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:21.451319 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:11:21.453098 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:21.453123 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:21.453590 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:21.454004 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:21.454611 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:21.455924 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:21.457423 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:21.458143 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:21.458342 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:21.459567 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:21.459575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:21.459591 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:21.460158 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:21.460250 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:21.460283 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:21.460412 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:21.460673 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 18:11:21.461002 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":383,"threads_started":1}
I20260807 18:11:21.465457 30066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832: No bootstrap required, opened a new log
I20260807 18:11:21.466531 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:21.475034 30066 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } }
I20260807 18:11:21.475492 30066 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:21.475632 30066 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0487d7514794c4593f7f6fcd8503832, State: Initialized, Role: FOLLOWER
I20260807 18:11:21.476325 30066 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } }
I20260807 18:11:21.476613 30066 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:21.476770 30066 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:21.476965 30066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:21.477895 30069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:21.471121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46905 (local address 127.0.0.1:46457)
0807 18:11:21.471290 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:21.471296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:21.471933 (+ 637us) server_negotiation.cc:407] Connection header received
0807 18:11:21.472194 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:21.472199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:21.472275 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:21.472379 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:21.472986 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:21.473784 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:21.475157 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:21.475755 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:21.475895 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:21.477089 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:21.477096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:21.477102 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:21.477128 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:21.477181 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:21.477204 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:21.477325 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:21.477433 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 18:11:21.477539 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 18:11:21.479827 30066 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } }
I20260807 18:11:21.480363 30066 leader_election.cc:302] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0487d7514794c4593f7f6fcd8503832; no voters:
I20260807 18:11:21.480978 30066 leader_election.cc:288] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:21.481128 30076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:21.481855 30076 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [term 1 LEADER]: Becoming Leader. State: Replica: c0487d7514794c4593f7f6fcd8503832, State: Running, Role: LEADER
I20260807 18:11:21.482455 30076 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } }
I20260807 18:11:21.483203 30066 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:21.485251 30077 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0487d7514794c4593f7f6fcd8503832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } } }
I20260807 18:11:21.485697 30077 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:21.486768 30083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:21.487357 30078 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0487d7514794c4593f7f6fcd8503832. Latest consensus state: current_term: 1 leader_uuid: "c0487d7514794c4593f7f6fcd8503832" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0487d7514794c4593f7f6fcd8503832" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46457 } } }
I20260807 18:11:21.487820 30078 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:21.492847 30083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:21.503176 30083 catalog_manager.cc:1347] Generated new cluster ID: 6eebcee6d6df4062b9640423e4f2c861
I20260807 18:11:21.503337 30083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:21.528043 30083 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:21.530160 30083 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:21.537406 30083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0487d7514794c4593f7f6fcd8503832: Generated new TSK 0
I20260807 18:11:21.538465 30083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:21.546211 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29998
W20260807 18:11:21.547598 30061 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40035
W20260807 18:11:21.548388 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 29998: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:21.548661 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 29998
2026-08-07T18:11:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2003 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:21 dist-test-slave-rnrw krb5kdc[30098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:21 dist-test-slave-rnrw krb5kdc[30098](info): set up 2 sockets
Aug 07 18:11:21 dist-test-slave-rnrw krb5kdc[30098](info): commencing operation
krb5kdc: starting...
W20260807 18:11:22.715306 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:22 dist-test-slave-rnrw krb5kdc[30098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:23.016829 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:23.371487 30112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:23.371826 30112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:23.371923 30112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:23.379747 30112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:23.379887 30112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:23.379936 30112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:23.380005 30112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:23.380054 30112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:23.390842 30112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37655
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:23.392148 30112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:23.393558 30112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:23.406160 30118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:23.406277 30117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:23.406205 30120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:23.410490 30112 server_base.cc:1034] running on GCE node
I20260807 18:11:23.411366 30112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:23.413021 30112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:23.414287 30112 hybrid_clock.cc:648] HybridClock initialized: now 1786126283414240 us; error 43 us; skew 500 ppm
I20260807 18:11:23.414700 30112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:23 dist-test-slave-rnrw krb5kdc[30098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126283, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:23.423522 30112 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:23.425500 30112 webserver.cc:460] Webserver started at http://127.0.0.1:43311/ using document root <none> and password file <none>
I20260807 18:11:23.426312 30112 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:23.426465 30112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:23.426795 30112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:23.430027 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "e33f417c6b8e4d6287de0fda01356146"
format_stamp: "Formatted at 2026-08-07 18:11:23 on dist-test-slave-rnrw"
server_key: "4f993b7c79149d8fb95873f706cc08c7"
server_key_iv: "cb8d5c705071c345650f42e6926191a5"
server_key_version: "encryptionkey@0"
I20260807 18:11:23.430828 30112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "e33f417c6b8e4d6287de0fda01356146"
format_stamp: "Formatted at 2026-08-07 18:11:23 on dist-test-slave-rnrw"
server_key: "4f993b7c79149d8fb95873f706cc08c7"
server_key_iv: "cb8d5c705071c345650f42e6926191a5"
server_key_version: "encryptionkey@0"
I20260807 18:11:23.436619 30112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:11:23.440503 30127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:23.442093 30112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:23.442286 30112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "e33f417c6b8e4d6287de0fda01356146"
format_stamp: "Formatted at 2026-08-07 18:11:23 on dist-test-slave-rnrw"
server_key: "4f993b7c79149d8fb95873f706cc08c7"
server_key_iv: "cb8d5c705071c345650f42e6926191a5"
server_key_version: "encryptionkey@0"
I20260807 18:11:23.442523 30112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:23.448782 30112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:23.453531 30112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:23.453858 30112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:23.474870 30112 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37655
I20260807 18:11:23.474912 30179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37655 every 8 connection(s)
I20260807 18:11:23.476663 30112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:23.480479 30180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:23.486060 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30112
I20260807 18:11:23.486260 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:23.486763 8442 external_mini_cluster.cc:1428] Setting key 65b31156533eb7a5937259dd2ce622ed
I20260807 18:11:23.491479 30180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146: Bootstrap starting.
I20260807 18:11:23.495213 30180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:23.495491 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:23.488872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60645 (local address 127.0.0.1:37655)
0807 18:11:23.489871 (+ 999us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:23.489883 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:23.489927 (+ 44us) server_negotiation.cc:407] Connection header received
0807 18:11:23.491823 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:23.491847 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:23.492220 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:23.492623 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:23.493775 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:23.493782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:23.493795 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:23.493811 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:23.493879 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:23.493903 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:23.494097 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:23.494206 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 18:11:23.494489 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":485,"threads_started":1}
I20260807 18:11:23.496383 30180 log.cc:824] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:23.499168 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:23.499466 30180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146: No bootstrap required, opened a new log
I20260807 18:11:23.506829 30184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:23.503911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58207 (local address 127.0.0.1:37655)
0807 18:11:23.504165 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:23.504176 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:23.504765 (+ 589us) server_negotiation.cc:407] Connection header received
0807 18:11:23.504980 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:23.504984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:23.505035 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:23.505167 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:23.506129 (+ 962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:23.506135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:23.506140 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:23.506160 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:23.506214 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:23.506234 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:23.506381 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:23.506486 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 18:11:23.506557 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:11:23.507732 30180 raft_consensus.cc:348] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } }
I20260807 18:11:23.508301 30180 raft_consensus.cc:374] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:23.508416 30180 raft_consensus.cc:729] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e33f417c6b8e4d6287de0fda01356146, State: Initialized, Role: FOLLOWER
I20260807 18:11:23.509262 30180 consensus_queue.cc:260] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } }
I20260807 18:11:23.509523 30180 raft_consensus.cc:388] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:23.509631 30180 raft_consensus.cc:482] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:23.509760 30180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:23.511899 30180 raft_consensus.cc:504] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } }
I20260807 18:11:23.512375 30180 leader_election.cc:302] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e33f417c6b8e4d6287de0fda01356146; no voters:
I20260807 18:11:23.513085 30180 leader_election.cc:288] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:23.513236 30191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:23.514530 30191 raft_consensus.cc:686] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [term 1 LEADER]: Becoming Leader. State: Replica: e33f417c6b8e4d6287de0fda01356146, State: Running, Role: LEADER
I20260807 18:11:23.515254 30191 consensus_queue.cc:237] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } }
I20260807 18:11:23.515447 30180 sys_catalog.cc:564] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:23.517925 30193 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e33f417c6b8e4d6287de0fda01356146. Latest consensus state: current_term: 1 leader_uuid: "e33f417c6b8e4d6287de0fda01356146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } } }
I20260807 18:11:23.518309 30193 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:23.519177 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e33f417c6b8e4d6287de0fda01356146" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e33f417c6b8e4d6287de0fda01356146" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37655 } } }
I20260807 18:11:23.519519 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:23.519526 30198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:23.532156 30198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:23.542968 30198 catalog_manager.cc:1347] Generated new cluster ID: 727e8a051c17498c8848f2f6e536fa56
I20260807 18:11:23.543120 30198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:23.555425 30198 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:23.555874 30198 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:23.563122 30198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e33f417c6b8e4d6287de0fda01356146: Generated new TSK 0
I20260807 18:11:23.563985 30198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:23.579754 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30112
W20260807 18:11:23.581017 30175 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60645
W20260807 18:11:23.581847 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:23.582074 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30112
2026-08-07T18:11:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2036 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:23 dist-test-slave-rnrw krb5kdc[30213](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:23 dist-test-slave-rnrw krb5kdc[30213](info): set up 2 sockets
Aug 07 18:11:23 dist-test-slave-rnrw krb5kdc[30213](info): commencing operation
krb5kdc: starting...
W20260807 18:11:24.759642 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:24 dist-test-slave-rnrw krb5kdc[30213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:25.055552 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:25.404841 30226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:25.405154 30226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:25.405308 30226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:25.412838 30226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:25.412949 30226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:25.413035 30226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:25.413141 30226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:25.413215 30226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:25.424335 30226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:25.425786 30226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:25.427131 30226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:25.439850 30234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:25.439888 30232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:25.439849 30231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:25.440958 30226 server_base.cc:1034] running on GCE node
I20260807 18:11:25.441852 30226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:25.443302 30226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:25.444684 30226 hybrid_clock.cc:648] HybridClock initialized: now 1786126285444519 us; error 50 us; skew 500 ppm
I20260807 18:11:25.445111 30226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:25 dist-test-slave-rnrw krb5kdc[30213](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126285, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:25.448673 30226 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:25.450379 30226 webserver.cc:460] Webserver started at http://127.0.0.1:40307/ using document root <none> and password file <none>
I20260807 18:11:25.451153 30226 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:25.451308 30226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:25.451654 30226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:25.454921 30226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "fb1f8f7fe62e4657aa9543ebb073c55e"
format_stamp: "Formatted at 2026-08-07 18:11:25 on dist-test-slave-rnrw"
server_key: "f68effc3e11c82d590e6403edf45855e"
server_key_iv: "1e9922c840b5e40c533932f003d1feb4"
server_key_version: "encryptionkey@0"
I20260807 18:11:25.455772 30226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "fb1f8f7fe62e4657aa9543ebb073c55e"
format_stamp: "Formatted at 2026-08-07 18:11:25 on dist-test-slave-rnrw"
server_key: "f68effc3e11c82d590e6403edf45855e"
server_key_iv: "1e9922c840b5e40c533932f003d1feb4"
server_key_version: "encryptionkey@0"
I20260807 18:11:25.461418 30226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 18:11:25.465268 30241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:25.466657 30226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:11:25.466862 30226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "fb1f8f7fe62e4657aa9543ebb073c55e"
format_stamp: "Formatted at 2026-08-07 18:11:25 on dist-test-slave-rnrw"
server_key: "f68effc3e11c82d590e6403edf45855e"
server_key_iv: "1e9922c840b5e40c533932f003d1feb4"
server_key_version: "encryptionkey@0"
I20260807 18:11:25.467073 30226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:25.495599 30226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:25.500177 30226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:25.500504 30226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:25.521421 30226 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34053
I20260807 18:11:25.521456 30295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34053 every 8 connection(s)
I20260807 18:11:25.522921 30226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:25.523922 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30226
I20260807 18:11:25.524134 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:25.524709 8442 external_mini_cluster.cc:1428] Setting key dca4d5e9cb36a8ffbacc6a14f56faf74
I20260807 18:11:25.527796 30296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 18:11:25 dist-test-slave-rnrw krb5kdc[30213](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:11:25.539021 30296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e: Bootstrap starting.
I20260807 18:11:25.542609 30296 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:25.543690 30296 log.cc:824] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:25.547197 30296 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e: No bootstrap required, opened a new log
I20260807 18:11:25.548444 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:25.526713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:57111 (local address 127.0.0.1:34053)
0807 18:11:25.527510 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:25.527524 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:25.527566 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:11:25.529359 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:25.529381 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:25.529764 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:25.530161 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:25.531647 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:25.533060 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:25.534509 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:25.535182 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:25.535399 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:25.540062 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:25.540070 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:25.540085 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:25.540124 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:25.543488 (+ 3364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:25.544417 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:25.544424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:25.544430 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:25.544665 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:25.545220 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:25.545228 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:25.545234 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:25.545918 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:25.546263 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:25.546711 (+ 448us) server_negotiation.cc:299] Negotiation successful
0807 18:11:25.547062 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":319,"threads_started":1}
I20260807 18:11:25.552510 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:25.556082 30296 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260807 18:11:25.556456 30296 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:25.556623 30296 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb1f8f7fe62e4657aa9543ebb073c55e, State: Initialized, Role: FOLLOWER
I20260807 18:11:25.557329 30296 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260807 18:11:25.557603 30296 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:25.557755 30296 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:25.557971 30296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:25.560783 30296 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260807 18:11:25.561342 30296 leader_election.cc:302] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb1f8f7fe62e4657aa9543ebb073c55e; no voters:
I20260807 18:11:25.562053 30296 leader_election.cc:288] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:25.562160 30306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:25.562855 30306 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [term 1 LEADER]: Becoming Leader. State: Replica: fb1f8f7fe62e4657aa9543ebb073c55e, State: Running, Role: LEADER
I20260807 18:11:25.563578 30306 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } }
I20260807 18:11:25.565711 30296 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:25.567937 30307 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb1f8f7fe62e4657aa9543ebb073c55e. Latest consensus state: current_term: 1 leader_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } } }
I20260807 18:11:25.568343 30307 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:25.578575 30299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:25.557131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:46835 (local address 127.0.0.1:34053)
0807 18:11:25.557322 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:25.557329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:25.557764 (+ 435us) server_negotiation.cc:407] Connection header received
0807 18:11:25.558010 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:25.558016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:25.558108 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:25.558227 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:25.559626 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:25.560771 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:25.562089 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:25.562778 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:25.562940 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:25.565938 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:25.565944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:25.565949 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:25.565991 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:25.572381 (+ 6390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:25.573954 (+ 1573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:25.573962 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:25.573969 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:25.574192 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:25.574982 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:25.574989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:25.574995 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:25.575508 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:25.577491 (+ 1983us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:25.577741 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 18:11:25.577995 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 18:11:25.570154 30308 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1f8f7fe62e4657aa9543ebb073c55e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34053 } } }
I20260807 18:11:25.579104 30308 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:25.583335 30313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:25.590583 30313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:25.600019 30313 catalog_manager.cc:1347] Generated new cluster ID: ed79073784604d08bd3b8a1eb32a9e05
I20260807 18:11:25.600150 30313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:25.624895 30313 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:25.626237 30313 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:25.632165 30313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb1f8f7fe62e4657aa9543ebb073c55e: Generated new TSK 0
I20260807 18:11:25.633150 30313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:25.644657 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30226
I20260807 18:11:25.645766 30291 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:25.738675 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30226
W20260807 18:11:25.758546 30211 connection.cc:535] client connection to 127.0.0.1:34053 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34053): Connection reset by peer (error 104)
2026-08-07T18:11:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2156 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:25 dist-test-slave-rnrw krb5kdc[30328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:25 dist-test-slave-rnrw krb5kdc[30328](info): set up 2 sockets
Aug 07 18:11:25 dist-test-slave-rnrw krb5kdc[30328](info): commencing operation
krb5kdc: starting...
W20260807 18:11:26.875806 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:27 dist-test-slave-rnrw krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:27.171849 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39591
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39591
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:27.522527 30342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:27.522840 30342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:27.523000 30342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:27.530226 30342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:27.530340 30342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:27.530467 30342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:27.530565 30342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:27.530646 30342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:27.540988 30342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:27.542354 30342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:27.543727 30342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:27.555593 30347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:27.555729 30348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:27.555857 30350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:27.556852 30342 server_base.cc:1034] running on GCE node
I20260807 18:11:27.557642 30342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:27.559056 30342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:27.560259 30342 hybrid_clock.cc:648] HybridClock initialized: now 1786126287560208 us; error 40 us; skew 500 ppm
I20260807 18:11:27.560657 30342 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:27 dist-test-slave-rnrw krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126287, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:27.563846 30342 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:27.565366 30342 webserver.cc:460] Webserver started at http://127.0.0.1:33841/ using document root <none> and password file <none>
I20260807 18:11:27.566051 30342 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:27.566162 30342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:27.566409 30342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:27.569514 30342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "6086289859114b1fbe1dd019da02572d"
format_stamp: "Formatted at 2026-08-07 18:11:27 on dist-test-slave-rnrw"
server_key: "91f2f67fa3d1c9a6a922a91e5fd56b3a"
server_key_iv: "2f5ba8183fdd72df33f9a6994e59ef93"
server_key_version: "encryptionkey@0"
I20260807 18:11:27.570317 30342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "6086289859114b1fbe1dd019da02572d"
format_stamp: "Formatted at 2026-08-07 18:11:27 on dist-test-slave-rnrw"
server_key: "91f2f67fa3d1c9a6a922a91e5fd56b3a"
server_key_iv: "2f5ba8183fdd72df33f9a6994e59ef93"
server_key_version: "encryptionkey@0"
I20260807 18:11:27.575546 30342 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 18:11:27.579142 30357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:27.580487 30342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 18:11:27.580760 30342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "6086289859114b1fbe1dd019da02572d"
format_stamp: "Formatted at 2026-08-07 18:11:27 on dist-test-slave-rnrw"
server_key: "91f2f67fa3d1c9a6a922a91e5fd56b3a"
server_key_iv: "2f5ba8183fdd72df33f9a6994e59ef93"
server_key_version: "encryptionkey@0"
I20260807 18:11:27.580971 30342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:27.598436 30342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:27.602872 30342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:27.603178 30342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:27.622424 30342 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39591
I20260807 18:11:27.622465 30411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39591 every 8 connection(s)
I20260807 18:11:27.623891 30342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:27.627904 30412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:27.631124 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30342
I20260807 18:11:27.631304 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:27.631757 8442 external_mini_cluster.cc:1428] Setting key bbd8dc5589fbe38c8308833475ff4110
I20260807 18:11:27.640066 30412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d: Bootstrap starting.
I20260807 18:11:27.643810 30412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:27.645087 30412 log.cc:824] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:27.645720 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:27.633565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:55143 (local address 127.0.0.1:39591)
0807 18:11:27.634327 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:27.634340 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:27.634381 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:11:27.636070 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:27.636092 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:27.636519 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:27.636915 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:27.637552 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:27.638893 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:27.640837 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:27.641576 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:27.641726 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:27.643133 (+ 1407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:27.643140 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:27.643155 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:27.643712 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:27.643794 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:27.643829 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:27.644019 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:27.644330 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 18:11:27.644495 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":347,"threads_started":1}
I20260807 18:11:27.647991 30412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d: No bootstrap required, opened a new log
I20260807 18:11:27.649559 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:27.658680 30412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260807 18:11:27.659214 30412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:27.659340 30412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6086289859114b1fbe1dd019da02572d, State: Initialized, Role: FOLLOWER
I20260807 18:11:27.659972 30412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260807 18:11:27.660238 30412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:27.660410 30412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:27.660634 30412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:27.662534 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:27.655101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:43763 (local address 127.0.0.1:39591)
0807 18:11:27.655261 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:27.655266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:27.655599 (+ 333us) server_negotiation.cc:407] Connection header received
0807 18:11:27.655796 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:27.655799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:27.655868 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:27.655950 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:27.656679 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:27.657667 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:27.659534 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:27.660171 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:27.660321 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:27.661679 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:27.661684 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:27.661687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:27.661710 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:27.661769 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:27.661792 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:27.661947 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:27.662159 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 18:11:27.662219 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 18:11:27.663218 30412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260807 18:11:27.663749 30412 leader_election.cc:302] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6086289859114b1fbe1dd019da02572d; no voters:
I20260807 18:11:27.664389 30412 leader_election.cc:288] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:27.664518 30422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:27.665381 30422 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [term 1 LEADER]: Becoming Leader. State: Replica: 6086289859114b1fbe1dd019da02572d, State: Running, Role: LEADER
I20260807 18:11:27.666141 30422 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } }
I20260807 18:11:27.666836 30412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:27.669075 30423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6086289859114b1fbe1dd019da02572d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } } }
I20260807 18:11:27.669435 30423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:27.674757 30424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6086289859114b1fbe1dd019da02572d. Latest consensus state: current_term: 1 leader_uuid: "6086289859114b1fbe1dd019da02572d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6086289859114b1fbe1dd019da02572d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39591 } } }
I20260807 18:11:27.675669 30424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:27.678311 30428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:27.683152 30428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:27.693012 30428 catalog_manager.cc:1347] Generated new cluster ID: 149946cd15a2423f9c88d8014194a8ea
I20260807 18:11:27.693146 30428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:27.722029 30428 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:27.723495 30428 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:27.741623 30428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6086289859114b1fbe1dd019da02572d: Generated new TSK 0
I20260807 18:11:27.742731 30428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:27.761008 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30342
W20260807 18:11:27.762346 30407 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.60:55143
W20260807 18:11:27.763180 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30342: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:27.763337 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30342
W20260807 18:11:27.782433 30326 connection.cc:535] client connection to 127.0.0.1:39591 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39591): Connection reset by peer (error 104)
2026-08-07T18:11:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:27 dist-test-slave-rnrw krb5kdc[30443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:27 dist-test-slave-rnrw krb5kdc[30443](info): set up 2 sockets
Aug 07 18:11:27 dist-test-slave-rnrw krb5kdc[30443](info): commencing operation
krb5kdc: starting...
W20260807 18:11:28.904417 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:29 dist-test-slave-rnrw krb5kdc[30443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126289, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:29.196197 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:29.547260 30456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:29.547659 30456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:29.547827 30456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:29.555394 30456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:29.555562 30456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:29.555663 30456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:29.555768 30456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:29.555845 30456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:29.566162 30456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:29.567590 30456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:29.568974 30456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:29.581106 30462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:29.581175 30463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:29.581202 30466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:29.582003 30456 server_base.cc:1034] running on GCE node
I20260807 18:11:29.582835 30456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:29.584296 30456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:29.585536 30456 hybrid_clock.cc:648] HybridClock initialized: now 1786126289585455 us; error 74 us; skew 500 ppm
I20260807 18:11:29.585914 30456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:29 dist-test-slave-rnrw krb5kdc[30443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126289, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:29.594513 30456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:29.596166 30456 webserver.cc:460] Webserver started at http://127.0.0.1:43279/ using document root <none> and password file <none>
I20260807 18:11:29.597018 30456 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:29.597180 30456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:29.597508 30456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:29.600765 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "362b096120594c12955aa32ac64686dd"
format_stamp: "Formatted at 2026-08-07 18:11:29 on dist-test-slave-rnrw"
server_key: "b3b56a9c10932ebc23b7570ad1ad4dab"
server_key_iv: "4fb38c94a35471dafb8132059de30e40"
server_key_version: "encryptionkey@0"
I20260807 18:11:29.601603 30456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "362b096120594c12955aa32ac64686dd"
format_stamp: "Formatted at 2026-08-07 18:11:29 on dist-test-slave-rnrw"
server_key: "b3b56a9c10932ebc23b7570ad1ad4dab"
server_key_iv: "4fb38c94a35471dafb8132059de30e40"
server_key_version: "encryptionkey@0"
I20260807 18:11:29.607599 30456 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 18:11:29.611752 30473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:29.613462 30456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:29.613682 30456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "362b096120594c12955aa32ac64686dd"
format_stamp: "Formatted at 2026-08-07 18:11:29 on dist-test-slave-rnrw"
server_key: "b3b56a9c10932ebc23b7570ad1ad4dab"
server_key_iv: "4fb38c94a35471dafb8132059de30e40"
server_key_version: "encryptionkey@0"
I20260807 18:11:29.613909 30456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:29.620071 30456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:29.624795 30456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:29.625156 30456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:29.644629 30456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38205
I20260807 18:11:29.644681 30524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38205 every 8 connection(s)
I20260807 18:11:29.646085 30456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:29.650190 30525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:29.655695 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30456
I20260807 18:11:29.655906 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:29.656456 8442 external_mini_cluster.cc:1428] Setting key 999f40b63ab90496099d7d20fb876781
I20260807 18:11:29.662117 30525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd: Bootstrap starting.
I20260807 18:11:29.664722 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:29.658424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:43445 (local address 127.0.0.1:38205)
0807 18:11:29.659294 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:29.659307 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:29.659352 (+ 45us) server_negotiation.cc:407] Connection header received
0807 18:11:29.661186 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:29.661210 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:29.661571 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:29.661920 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:29.663026 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:29.663032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:29.663045 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:29.663058 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:29.663112 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:29.663131 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:29.663295 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:29.663455 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 18:11:29.663737 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":397,"threads_started":1}
I20260807 18:11:29.665720 30525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:29.667160 30525 log.cc:824] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:29.668776 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:29.670217 30525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd: No bootstrap required, opened a new log
I20260807 18:11:29.675793 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:29.673024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:44307 (local address 127.0.0.1:38205)
0807 18:11:29.673250 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:29.673259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:29.673681 (+ 422us) server_negotiation.cc:407] Connection header received
0807 18:11:29.673865 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:29.673869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:29.673918 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:29.674010 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:29.675050 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:29.675056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:29.675063 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:29.675079 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:29.675122 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:29.675138 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:29.675245 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:29.675371 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 18:11:29.675431 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 18:11:29.678988 30525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260807 18:11:29.679404 30525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:29.679531 30525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b096120594c12955aa32ac64686dd, State: Initialized, Role: FOLLOWER
I20260807 18:11:29.680136 30525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260807 18:11:29.680378 30525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:29.680600 30525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:29.680787 30525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:29.683070 30525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260807 18:11:29.683601 30525 leader_election.cc:302] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 362b096120594c12955aa32ac64686dd; no voters:
I20260807 18:11:29.684343 30525 leader_election.cc:288] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:29.684475 30535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:29.685248 30535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [term 1 LEADER]: Becoming Leader. State: Replica: 362b096120594c12955aa32ac64686dd, State: Running, Role: LEADER
I20260807 18:11:29.685866 30535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } }
I20260807 18:11:29.686733 30525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:29.689005 30536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 362b096120594c12955aa32ac64686dd. Latest consensus state: current_term: 1 leader_uuid: "362b096120594c12955aa32ac64686dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
I20260807 18:11:29.689049 30537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "362b096120594c12955aa32ac64686dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362b096120594c12955aa32ac64686dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38205 } } }
I20260807 18:11:29.689456 30536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:29.689504 30537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:29.696105 30542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:29.701598 30542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:29.711082 30542 catalog_manager.cc:1347] Generated new cluster ID: 96c6f44a11194c668e54bd6bec62e932
I20260807 18:11:29.711213 30542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:29.725534 30542 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:29.725960 30542 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:29.734592 30542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 362b096120594c12955aa32ac64686dd: Generated new TSK 0
I20260807 18:11:29.735519 30542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:29.751152 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30456
W20260807 18:11:29.752388 30520 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.60:43445
W20260807 18:11:29.753225 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30456: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:29.753389 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30456
2026-08-07T18:11:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1990 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:29 dist-test-slave-rnrw krb5kdc[30556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:29 dist-test-slave-rnrw krb5kdc[30556](info): set up 2 sockets
Aug 07 18:11:29 dist-test-slave-rnrw krb5kdc[30556](info): commencing operation
krb5kdc: starting...
W20260807 18:11:30.905185 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:31 dist-test-slave-rnrw krb5kdc[30556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:31.195719 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37897
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37897
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:31.553123 30569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:31.553522 30569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:31.553687 30569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:31.561306 30569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:31.561544 30569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:31.561658 30569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:31.561769 30569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:31.561857 30569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:31.572176 30569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37897
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:31.573693 30569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:31.575109 30569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:31.588160 30575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:31.588340 30574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:31.588083 30577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:31.589128 30569 server_base.cc:1034] running on GCE node
I20260807 18:11:31.590019 30569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:31.591580 30569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:31.592808 30569 hybrid_clock.cc:648] HybridClock initialized: now 1786126291592764 us; error 46 us; skew 500 ppm
I20260807 18:11:31.593165 30569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:31 dist-test-slave-rnrw krb5kdc[30556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126291, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:31.596658 30569 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:31.598201 30569 webserver.cc:460] Webserver started at http://127.0.0.1:46883/ using document root <none> and password file <none>
I20260807 18:11:31.598924 30569 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:31.599067 30569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:31.599365 30569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:31.602394 30569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "82115d2cbb054451b4e304c3f782569f"
format_stamp: "Formatted at 2026-08-07 18:11:31 on dist-test-slave-rnrw"
server_key: "c43aabf676c1c212242ef768a357ddbd"
server_key_iv: "6acd330e76f80a538732edcc8f073076"
server_key_version: "encryptionkey@0"
I20260807 18:11:31.603179 30569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "82115d2cbb054451b4e304c3f782569f"
format_stamp: "Formatted at 2026-08-07 18:11:31 on dist-test-slave-rnrw"
server_key: "c43aabf676c1c212242ef768a357ddbd"
server_key_iv: "6acd330e76f80a538732edcc8f073076"
server_key_version: "encryptionkey@0"
I20260807 18:11:31.608438 30569 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 18:11:31.612216 30584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:31.613804 30569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:31.614015 30569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "82115d2cbb054451b4e304c3f782569f"
format_stamp: "Formatted at 2026-08-07 18:11:31 on dist-test-slave-rnrw"
server_key: "c43aabf676c1c212242ef768a357ddbd"
server_key_iv: "6acd330e76f80a538732edcc8f073076"
server_key_version: "encryptionkey@0"
I20260807 18:11:31.614233 30569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:31.659340 30569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:31.663899 30569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:31.664232 30569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:31.684710 30569 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37897
I20260807 18:11:31.684715 30638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37897 every 8 connection(s)
I20260807 18:11:31.686374 30569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:31.690371 30639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:31.696034 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30569
I20260807 18:11:31.696252 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:31.696746 8442 external_mini_cluster.cc:1428] Setting key ee1081dc5cebe8380e04dd42897df797
I20260807 18:11:31.701902 30639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f: Bootstrap starting.
I20260807 18:11:31.705852 30639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:31.707180 30639 log.cc:824] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:31.710865 30639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f: No bootstrap required, opened a new log
Aug 07 18:11:31 dist-test-slave-rnrw krb5kdc[30556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 18:11:31.721520 30639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } }
I20260807 18:11:31.722016 30639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:31.722153 30639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82115d2cbb054451b4e304c3f782569f, State: Initialized, Role: FOLLOWER
I20260807 18:11:31.722765 30639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } }
I20260807 18:11:31.723026 30639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:31.723213 30639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:31.723263 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:31.698644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:51511 (local address 127.0.0.1:37897)
0807 18:11:31.699583 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:31.699598 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:31.699639 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:11:31.701468 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:31.701494 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:31.701908 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:31.702284 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:31.704242 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:31.705696 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:31.707640 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:31.708844 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:31.708981 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:31.713985 (+ 5004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:31.714011 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:31.714032 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:31.714089 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:31.718262 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:31.719255 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:31.719265 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:31.719271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:31.719509 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:31.720122 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:31.720126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:31.720128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:31.720773 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:31.720997 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:31.721599 (+ 602us) server_negotiation.cc:299] Negotiation successful
0807 18:11:31.721930 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":473,"threads_started":1}
I20260807 18:11:31.723474 30639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:31.726414 30639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } }
I20260807 18:11:31.727001 30639 leader_election.cc:302] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82115d2cbb054451b4e304c3f782569f; no voters:
I20260807 18:11:31.727733 30639 leader_election.cc:288] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:31.727792 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:31.727838 30644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:31.730331 30639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:31.732932 30644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [term 1 LEADER]: Becoming Leader. State: Replica: 82115d2cbb054451b4e304c3f782569f, State: Running, Role: LEADER
I20260807 18:11:31.733706 30644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } }
I20260807 18:11:31.736892 30646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82115d2cbb054451b4e304c3f782569f. Latest consensus state: current_term: 1 leader_uuid: "82115d2cbb054451b4e304c3f782569f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } } }
I20260807 18:11:31.736984 30645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82115d2cbb054451b4e304c3f782569f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82115d2cbb054451b4e304c3f782569f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37897 } } }
I20260807 18:11:31.737327 30646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:31.737394 30645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:31.743659 30657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:31.750589 30657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:31.752251 30642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:31.732642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:48183 (local address 127.0.0.1:37897)
0807 18:11:31.732848 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:31.732856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:31.733420 (+ 564us) server_negotiation.cc:407] Connection header received
0807 18:11:31.733622 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:31.733627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:31.733704 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:31.733806 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:31.735316 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:31.736690 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:31.738971 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:31.739934 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:31.740080 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:31.744153 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:31.744159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:31.744162 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:31.744196 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:31.747634 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:31.748670 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:31.748677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:31.748683 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:31.748840 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:31.749475 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:31.749483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:31.749488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:31.750038 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:31.750334 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:31.751688 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0807 18:11:31.751878 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:31.762027 30657 catalog_manager.cc:1347] Generated new cluster ID: 9068a15ccc33470a8b9a2581f167ae5d
I20260807 18:11:31.762177 30657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:31.776799 30657 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:31.778059 30657 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:31.789592 30657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82115d2cbb054451b4e304c3f782569f: Generated new TSK 0
I20260807 18:11:31.790495 30657 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:31.798347 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30569
I20260807 18:11:31.799562 30634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:31.892959 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30569
W20260807 18:11:31.913028 30554 connection.cc:535] client connection to 127.0.0.1:37897 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37897): Connection reset by peer (error 104)
2026-08-07T18:11:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2140 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:32 dist-test-slave-rnrw krb5kdc[30671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:32 dist-test-slave-rnrw krb5kdc[30671](info): set up 2 sockets
Aug 07 18:11:32 dist-test-slave-rnrw krb5kdc[30671](info): commencing operation
krb5kdc: starting...
W20260807 18:11:33.071404 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:33 dist-test-slave-rnrw krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126293, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:33.372462 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38645
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:33.726892 30685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:33.727202 30685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:33.727368 30685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:33.734776 30685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:33.734887 30685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:33.734992 30685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:33.735102 30685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:33.735165 30685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:33.749326 30685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38645
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:33.751163 30685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:33.753057 30685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:33.769865 30691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:33.769935 30693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:33.770200 30690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:33.770772 30685 server_base.cc:1034] running on GCE node
I20260807 18:11:33.772090 30685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:33.774554 30685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:33.775888 30685 hybrid_clock.cc:648] HybridClock initialized: now 1786126293775771 us; error 96 us; skew 500 ppm
I20260807 18:11:33.776490 30685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:33 dist-test-slave-rnrw krb5kdc[30671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126293, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:33.781329 30685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:33.782985 30685 webserver.cc:460] Webserver started at http://127.0.0.1:39977/ using document root <none> and password file <none>
I20260807 18:11:33.783790 30685 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:33.783928 30685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:33.784253 30685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:33.788872 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "6ffe2a12584549e6a5b5260acfc3829f"
format_stamp: "Formatted at 2026-08-07 18:11:33 on dist-test-slave-rnrw"
server_key: "134f96cb09485623c7e5d3adece20d13"
server_key_iv: "14924462ff814e16c6b1ffdf60733150"
server_key_version: "encryptionkey@0"
I20260807 18:11:33.789914 30685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "6ffe2a12584549e6a5b5260acfc3829f"
format_stamp: "Formatted at 2026-08-07 18:11:33 on dist-test-slave-rnrw"
server_key: "134f96cb09485623c7e5d3adece20d13"
server_key_iv: "14924462ff814e16c6b1ffdf60733150"
server_key_version: "encryptionkey@0"
I20260807 18:11:33.795583 30685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:33.799410 30700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:33.801105 30685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:33.801304 30685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "6ffe2a12584549e6a5b5260acfc3829f"
format_stamp: "Formatted at 2026-08-07 18:11:33 on dist-test-slave-rnrw"
server_key: "134f96cb09485623c7e5d3adece20d13"
server_key_iv: "14924462ff814e16c6b1ffdf60733150"
server_key_version: "encryptionkey@0"
I20260807 18:11:33.801519 30685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:33.825008 30685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:33.829866 30685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:33.830180 30685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:33.850224 30685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38645
I20260807 18:11:33.850241 30751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38645 every 8 connection(s)
I20260807 18:11:33.851708 30685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:33.852176 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30685
I20260807 18:11:33.852396 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:33.853112 8442 external_mini_cluster.cc:1428] Setting key 3965bce123627c09edcff987c6c82739
I20260807 18:11:33.856726 30752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:33.866925 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:33.855087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:32789 (local address 127.0.0.1:38645)
0807 18:11:33.855879 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:33.855892 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:33.855934 (+ 42us) server_negotiation.cc:407] Connection header received
0807 18:11:33.857769 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:33.857794 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:33.858235 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:33.858626 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:33.859229 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:33.860618 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:33.862097 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:33.862797 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:33.863005 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:33.864356 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:33.864363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:33.864377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:33.865054 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:33.865156 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:33.865183 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:33.865368 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:33.865657 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 18:11:33.865858 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":337,"threads_started":1}
I20260807 18:11:33.868757 30752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f: Bootstrap starting.
I20260807 18:11:33.871436 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:33.872669 30752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:33.873806 30752 log.cc:824] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:33.876981 30752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f: No bootstrap required, opened a new log
I20260807 18:11:33.886206 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:33.878753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:44793 (local address 127.0.0.1:38645)
0807 18:11:33.879011 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:33.879018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:33.879364 (+ 346us) server_negotiation.cc:407] Connection header received
0807 18:11:33.879635 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:33.879639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:33.879714 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:33.879815 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:33.880489 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:33.881429 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:33.883359 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:33.883843 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:33.883948 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:33.885326 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:33.885340 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:33.885343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:33.885364 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:33.885404 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:33.885419 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:33.885519 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:33.885760 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 18:11:33.885848 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:33.887526 30752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } }
I20260807 18:11:33.887962 30752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:33.888075 30752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffe2a12584549e6a5b5260acfc3829f, State: Initialized, Role: FOLLOWER
I20260807 18:11:33.888831 30752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } }
I20260807 18:11:33.889087 30752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:33.889245 30752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:33.889438 30752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:33.891752 30752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } }
I20260807 18:11:33.892319 30752 leader_election.cc:302] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ffe2a12584549e6a5b5260acfc3829f; no voters:
I20260807 18:11:33.893064 30752 leader_election.cc:288] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:33.893254 30762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:33.894376 30762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [term 1 LEADER]: Becoming Leader. State: Replica: 6ffe2a12584549e6a5b5260acfc3829f, State: Running, Role: LEADER
I20260807 18:11:33.895044 30762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } }
I20260807 18:11:33.895453 30752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:33.898075 30765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ffe2a12584549e6a5b5260acfc3829f. Latest consensus state: current_term: 1 leader_uuid: "6ffe2a12584549e6a5b5260acfc3829f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } } }
I20260807 18:11:33.898540 30765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:33.899011 30764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ffe2a12584549e6a5b5260acfc3829f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ffe2a12584549e6a5b5260acfc3829f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38645 } } }
I20260807 18:11:33.899286 30764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:33.904874 30772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:33.910629 30772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:33.921185 30772 catalog_manager.cc:1347] Generated new cluster ID: a475714a084f4a5d850f3851937a5bcf
I20260807 18:11:33.921353 30772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:33.938063 30772 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:33.939493 30772 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:33.951025 30772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ffe2a12584549e6a5b5260acfc3829f: Generated new TSK 0
I20260807 18:11:33.951987 30772 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:33.966516 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30685
W20260807 18:11:33.968510 30747 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.60:32789
W20260807 18:11:33.969391 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30685: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:33.969681 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30685
W20260807 18:11:33.988833 30669 connection.cc:535] client connection to 127.0.0.1:38645 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38645): Connection reset by peer (error 104)
2026-08-07T18:11:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2075 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:34 dist-test-slave-rnrw krb5kdc[30785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:34 dist-test-slave-rnrw krb5kdc[30785](info): set up 2 sockets
Aug 07 18:11:34 dist-test-slave-rnrw krb5kdc[30785](info): commencing operation
krb5kdc: starting...
W20260807 18:11:35.163663 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.009s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:35 dist-test-slave-rnrw krb5kdc[30785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126295, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 18:11:35.456290 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:35.798900 30799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:35.799191 30799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:35.799352 30799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:35.806663 30799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:35.806784 30799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:35.806890 30799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:35.807000 30799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:35.807062 30799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:35.817090 30799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:35.818398 30799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:35.819749 30799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:35.832188 30805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:35.832298 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:35.832257 30804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:35.832563 30799 server_base.cc:1034] running on GCE node
I20260807 18:11:35.833693 30799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:35.835127 30799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:35.836395 30799 hybrid_clock.cc:648] HybridClock initialized: now 1786126295836323 us; error 58 us; skew 500 ppm
I20260807 18:11:35.836823 30799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:35 dist-test-slave-rnrw krb5kdc[30785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126295, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:35.846422 30799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 18:11:35.848022 30799 webserver.cc:460] Webserver started at http://127.0.0.1:38731/ using document root <none> and password file <none>
I20260807 18:11:35.848855 30799 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:35.849007 30799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:35.849344 30799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:35.852475 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da"
format_stamp: "Formatted at 2026-08-07 18:11:35 on dist-test-slave-rnrw"
server_key: "ecebcb5549682242b138b5111c4f6098"
server_key_iv: "b6133d59228f21c9f31ccf9bbd9340c5"
server_key_version: "encryptionkey@0"
I20260807 18:11:35.853468 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da"
format_stamp: "Formatted at 2026-08-07 18:11:35 on dist-test-slave-rnrw"
server_key: "ecebcb5549682242b138b5111c4f6098"
server_key_iv: "b6133d59228f21c9f31ccf9bbd9340c5"
server_key_version: "encryptionkey@0"
I20260807 18:11:35.858999 30799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:35.862843 30814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:35.864473 30799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 18:11:35.864732 30799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da"
format_stamp: "Formatted at 2026-08-07 18:11:35 on dist-test-slave-rnrw"
server_key: "ecebcb5549682242b138b5111c4f6098"
server_key_iv: "b6133d59228f21c9f31ccf9bbd9340c5"
server_key_version: "encryptionkey@0"
I20260807 18:11:35.864959 30799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:35.871441 30799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:35.876004 30799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:35.876349 30799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:35.896438 30799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39369
I20260807 18:11:35.896556 30865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39369 every 8 connection(s)
I20260807 18:11:35.898072 30799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:35.902271 30866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:35.904680 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30799
I20260807 18:11:35.904913 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:35.905385 8442 external_mini_cluster.cc:1428] Setting key c6c1e17f634208689b129f3b36654ab2
I20260807 18:11:35.915688 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:35.907573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:46141 (local address 127.0.0.1:39369)
0807 18:11:35.908621 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:35.908640 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:35.908701 (+ 61us) server_negotiation.cc:407] Connection header received
0807 18:11:35.911234 (+ 2533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:35.911269 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:35.911785 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:35.912300 (+ 515us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:35.913462 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:35.913470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:35.913490 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:35.913508 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:35.913593 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:35.913622 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:35.913875 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:35.914026 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 18:11:35.914276 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":478,"threads_started":1}
I20260807 18:11:35.917634 30866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da: Bootstrap starting.
I20260807 18:11:35.920063 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:35.921367 30866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:35.922487 30866 log.cc:824] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:35.926159 30866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da: No bootstrap required, opened a new log
I20260807 18:11:35.927619 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:35.924982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.60:38177 (local address 127.0.0.1:39369)
0807 18:11:35.925174 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:35.925181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:35.925529 (+ 348us) server_negotiation.cc:407] Connection header received
0807 18:11:35.925676 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:35.925682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:35.925742 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:35.925861 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:35.926913 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:35.926918 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:35.926922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:35.926943 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:35.926997 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:35.927055 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:35.927199 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:35.927256 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 18:11:35.927323 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:35.934727 30866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260807 18:11:35.935215 30866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:35.935361 30866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e7e8ea2d81945a7a6d63f7f75cb54da, State: Initialized, Role: FOLLOWER
I20260807 18:11:35.935971 30866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260807 18:11:35.936185 30866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:35.936290 30866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:35.936407 30866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:35.939098 30866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260807 18:11:35.939580 30866 leader_election.cc:302] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e7e8ea2d81945a7a6d63f7f75cb54da; no voters:
I20260807 18:11:35.940166 30866 leader_election.cc:288] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:35.940454 30876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:35.941251 30876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [term 1 LEADER]: Becoming Leader. State: Replica: 9e7e8ea2d81945a7a6d63f7f75cb54da, State: Running, Role: LEADER
I20260807 18:11:35.941803 30876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } }
I20260807 18:11:35.942430 30866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:35.944571 30878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e7e8ea2d81945a7a6d63f7f75cb54da. Latest consensus state: current_term: 1 leader_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } } }
I20260807 18:11:35.945029 30878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:35.944988 30877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e7e8ea2d81945a7a6d63f7f75cb54da" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39369 } } }
I20260807 18:11:35.945390 30877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:35.946894 30883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:35.952347 30883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:35.961372 30883 catalog_manager.cc:1347] Generated new cluster ID: 786d429c21df47adb8161b09de228da5
I20260807 18:11:35.961506 30883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:35.994415 30883 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:35.994951 30883 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:36.009116 30883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e7e8ea2d81945a7a6d63f7f75cb54da: Generated new TSK 0
I20260807 18:11:36.010118 30883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:36.019748 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30799
W20260807 18:11:36.021183 30861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.60:46141
W20260807 18:11:36.021957 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30799: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:36.022143 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30799
2026-08-07T18:11:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2052 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:36 dist-test-slave-rnrw krb5kdc[30897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:36 dist-test-slave-rnrw krb5kdc[30897](info): set up 2 sockets
Aug 07 18:11:36 dist-test-slave-rnrw krb5kdc[30897](info): commencing operation
krb5kdc: starting...
W20260807 18:11:37.167882 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:37 dist-test-slave-rnrw krb5kdc[30897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:37.468148 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36415
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:37.819496 30910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:37.819802 30910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:37.819965 30910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:37.827400 30910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:37.827550 30910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:37.827661 30910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:37.827764 30910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:37.827842 30910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:37.838202 30910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:36415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:37.839570 30910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:37.840972 30910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:37.853377 30918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:37.853559 30916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:37.853550 30915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:37.854533 30910 server_base.cc:1034] running on GCE node
I20260807 18:11:37.855584 30910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:37.857362 30910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:37.858630 30910 hybrid_clock.cc:648] HybridClock initialized: now 1786126297858563 us; error 70 us; skew 500 ppm
I20260807 18:11:37.859063 30910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:37 dist-test-slave-rnrw krb5kdc[30897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126297, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:37.862834 30910 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:37.864692 30910 webserver.cc:460] Webserver started at http://127.8.62.190:33913/ using document root <none> and password file <none>
I20260807 18:11:37.865603 30910 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:37.865758 30910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:37.866113 30910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:37.869619 30910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "05581b620bc144139b9b21a6941812d9"
format_stamp: "Formatted at 2026-08-07 18:11:37 on dist-test-slave-rnrw"
server_key: "c80d8e6255bb3436a908d369df9a8140"
server_key_iv: "f7bc5bd01fa548b492ead853c1602028"
server_key_version: "encryptionkey@0"
I20260807 18:11:37.870517 30910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "05581b620bc144139b9b21a6941812d9"
format_stamp: "Formatted at 2026-08-07 18:11:37 on dist-test-slave-rnrw"
server_key: "c80d8e6255bb3436a908d369df9a8140"
server_key_iv: "f7bc5bd01fa548b492ead853c1602028"
server_key_version: "encryptionkey@0"
I20260807 18:11:37.876718 30910 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 18:11:37.880949 30926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:37.882400 30910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:11:37.882648 30910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "05581b620bc144139b9b21a6941812d9"
format_stamp: "Formatted at 2026-08-07 18:11:37 on dist-test-slave-rnrw"
server_key: "c80d8e6255bb3436a908d369df9a8140"
server_key_iv: "f7bc5bd01fa548b492ead853c1602028"
server_key_version: "encryptionkey@0"
I20260807 18:11:37.882879 30910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:37.903261 30910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:37.908061 30910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:37.908389 30910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:37.930111 30910 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:36415
I20260807 18:11:37.930142 30979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:36415 every 8 connection(s)
I20260807 18:11:37.931609 30910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:37.935712 30980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:37.937211 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 30910
I20260807 18:11:37.937412 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:37.937847 8442 external_mini_cluster.cc:1428] Setting key e227a4487f911e1c8322f943f5b0ab6a
I20260807 18:11:37.952119 30980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9: Bootstrap starting.
Aug 07 18:11:37 dist-test-slave-rnrw krb5kdc[30897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:37.956653 30980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:37.958313 30980 log.cc:824] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:37.962020 30980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9: No bootstrap required, opened a new log
I20260807 18:11:37.965328 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:37.939922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47969 (local address 127.8.62.190:36415)
0807 18:11:37.940872 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:37.940884 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:37.940930 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:11:37.942588 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:37.942615 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:37.943052 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:37.943429 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:37.945503 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:37.946855 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:37.948896 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:37.949655 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:37.949799 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:37.955311 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:37.955325 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:37.955347 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:37.955415 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:37.959863 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:37.960993 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:37.961002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:37.961013 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:37.961255 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:37.961794 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:37.961800 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:37.961803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:37.962624 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:37.963037 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:37.963396 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 18:11:37.963778 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":419,"threads_started":1}
I20260807 18:11:37.969799 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:37.972909 30980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } }
I20260807 18:11:37.973387 30980 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:37.973507 30980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05581b620bc144139b9b21a6941812d9, State: Initialized, Role: FOLLOWER
I20260807 18:11:37.974273 30980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } }
I20260807 18:11:37.974746 30980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:37.974936 30980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:37.975153 30980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:37.978338 30980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } }
I20260807 18:11:37.978982 30980 leader_election.cc:302] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05581b620bc144139b9b21a6941812d9; no voters:
I20260807 18:11:37.979739 30980 leader_election.cc:288] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:37.979848 30990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:37.981230 30990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [term 1 LEADER]: Becoming Leader. State: Replica: 05581b620bc144139b9b21a6941812d9, State: Running, Role: LEADER
I20260807 18:11:37.981979 30990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } }
I20260807 18:11:37.982579 30980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:37.985461 30993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05581b620bc144139b9b21a6941812d9. Latest consensus state: current_term: 1 leader_uuid: "05581b620bc144139b9b21a6941812d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } } }
I20260807 18:11:37.985858 30993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:37.986321 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05581b620bc144139b9b21a6941812d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05581b620bc144139b9b21a6941812d9" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 36415 } } }
I20260807 18:11:37.986670 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:37.987488 30998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:37.993533 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:37.974509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37099 (local address 127.8.62.190:36415)
0807 18:11:37.974707 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:37.974713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:37.975234 (+ 521us) server_negotiation.cc:407] Connection header received
0807 18:11:37.975406 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:37.975412 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:37.975507 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:37.975638 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:37.977193 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:37.978404 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:37.980001 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:37.980839 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:37.980979 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:37.983999 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:37.984006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:37.984011 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:37.984053 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:37.988421 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:37.989503 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:37.989508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:37.989512 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:37.989665 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:37.990367 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:37.990375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:37.990381 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:37.990973 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:37.992593 (+ 1620us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:37.992826 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 18:11:37.993067 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 18:11:37.999567 30998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:38.011893 30998 catalog_manager.cc:1347] Generated new cluster ID: 3fbed83a81d344379a8b5efed0151f68
I20260807 18:11:38.012094 30998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:38.031494 30998 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:38.033528 30998 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:38.045918 30998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05581b620bc144139b9b21a6941812d9: Generated new TSK 0
I20260807 18:11:38.046998 30998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:38.059782 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 30910
I20260807 18:11:38.061895 30975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:38.157179 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 30910
W20260807 18:11:38.177073 30895 connection.cc:535] client connection to 127.8.62.190:36415 recv error: Network error: failed to read from TLS socket (remote: 127.8.62.190:36415): Connection reset by peer (error 104)
2026-08-07T18:11:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2136 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:38 dist-test-slave-rnrw krb5kdc[31013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:38 dist-test-slave-rnrw krb5kdc[31013](info): set up 2 sockets
Aug 07 18:11:38 dist-test-slave-rnrw krb5kdc[31013](info): commencing operation
krb5kdc: starting...
W20260807 18:11:39.308310 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:39 dist-test-slave-rnrw krb5kdc[31013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:39.601512 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:34253
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:34253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:39.948906 31026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:39.949199 31026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:39.949379 31026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:39.956753 31026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:39.956864 31026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:39.956977 31026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:39.957082 31026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:39.957171 31026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:39.966974 31026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:34253
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:34253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:39.968282 31026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:39.969630 31026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:39.981904 31032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:39.982342 31031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:39.983202 31026 server_base.cc:1034] running on GCE node
W20260807 18:11:39.983762 31034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:39.984683 31026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:39.986241 31026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:39.987504 31026 hybrid_clock.cc:648] HybridClock initialized: now 1786126299987461 us; error 41 us; skew 500 ppm
I20260807 18:11:39.987897 31026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:39 dist-test-slave-rnrw krb5kdc[31013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126299, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:39.991333 31026 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:39.992875 31026 webserver.cc:460] Webserver started at http://127.8.62.190:44693/ using document root <none> and password file <none>
I20260807 18:11:39.993559 31026 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:39.993666 31026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:39.993920 31026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:39.997344 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "a49b465cdda84e3ab2866ec6fc28418a"
format_stamp: "Formatted at 2026-08-07 18:11:39 on dist-test-slave-rnrw"
server_key: "858c3bab7e0b78a37d4d02bbf57f86c4"
server_key_iv: "bcf0bdd9bbdc376af6d09741d027e0ca"
server_key_version: "encryptionkey@0"
I20260807 18:11:39.998235 31026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "a49b465cdda84e3ab2866ec6fc28418a"
format_stamp: "Formatted at 2026-08-07 18:11:39 on dist-test-slave-rnrw"
server_key: "858c3bab7e0b78a37d4d02bbf57f86c4"
server_key_iv: "bcf0bdd9bbdc376af6d09741d027e0ca"
server_key_version: "encryptionkey@0"
I20260807 18:11:40.003985 31026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:40.008078 31041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:40.009732 31026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:40.009936 31026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "a49b465cdda84e3ab2866ec6fc28418a"
format_stamp: "Formatted at 2026-08-07 18:11:39 on dist-test-slave-rnrw"
server_key: "858c3bab7e0b78a37d4d02bbf57f86c4"
server_key_iv: "bcf0bdd9bbdc376af6d09741d027e0ca"
server_key_version: "encryptionkey@0"
I20260807 18:11:40.010164 31026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:40.024636 31026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:40.029299 31026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:40.029641 31026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:40.049427 31026 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:34253
I20260807 18:11:40.049460 31092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:34253 every 8 connection(s)
I20260807 18:11:40.050979 31026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:40.054803 31093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:40.060137 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31026
I20260807 18:11:40.060338 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:40.060868 8442 external_mini_cluster.cc:1428] Setting key afa611815421528957672891df55acee
I20260807 18:11:40.065891 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a: Bootstrap starting.
I20260807 18:11:40.069715 31093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:40.070895 31093 log.cc:824] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:40.074288 31093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a: No bootstrap required, opened a new log
I20260807 18:11:40.075016 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:40.062796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52497 (local address 127.8.62.190:34253)
0807 18:11:40.063717 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:40.063729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:40.063770 (+ 41us) server_negotiation.cc:407] Connection header received
0807 18:11:40.065549 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:40.065575 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:40.065995 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:40.066346 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:40.066981 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:40.068325 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:40.070231 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:40.070914 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:40.071134 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:40.072506 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:40.072515 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:40.072528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:40.073130 (+ 602us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:40.073207 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:40.073232 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:40.073360 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:40.073710 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 18:11:40.073900 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":436,"threads_started":1}
I20260807 18:11:40.079375 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:40.085116 31093 raft_consensus.cc:348] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } }
I20260807 18:11:40.085650 31093 raft_consensus.cc:374] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:40.085762 31093 raft_consensus.cc:729] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a49b465cdda84e3ab2866ec6fc28418a, State: Initialized, Role: FOLLOWER
I20260807 18:11:40.086512 31093 consensus_queue.cc:260] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } }
I20260807 18:11:40.086802 31093 raft_consensus.cc:388] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:40.086980 31093 raft_consensus.cc:482] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:40.087178 31093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:40.090453 31093 raft_consensus.cc:504] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } }
I20260807 18:11:40.091144 31093 leader_election.cc:302] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a49b465cdda84e3ab2866ec6fc28418a; no voters:
I20260807 18:11:40.092072 31093 leader_election.cc:288] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:40.092177 31104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:40.092484 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:40.084718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37291 (local address 127.8.62.190:34253)
0807 18:11:40.084945 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:40.084955 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:40.085465 (+ 510us) server_negotiation.cc:407] Connection header received
0807 18:11:40.085706 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:40.085710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:40.085814 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:40.085918 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:40.086485 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:40.087508 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:40.089433 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:40.089871 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:40.089964 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:40.091321 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:40.091329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:40.091334 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:40.091356 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:40.091422 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:40.091439 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:40.091522 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:40.091914 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 18:11:40.091988 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:11:40.096733 31104 raft_consensus.cc:686] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [term 1 LEADER]: Becoming Leader. State: Replica: a49b465cdda84e3ab2866ec6fc28418a, State: Running, Role: LEADER
I20260807 18:11:40.097489 31104 consensus_queue.cc:237] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } }
I20260807 18:11:40.098879 31093 sys_catalog.cc:564] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:40.100605 31106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a49b465cdda84e3ab2866ec6fc28418a. Latest consensus state: current_term: 1 leader_uuid: "a49b465cdda84e3ab2866ec6fc28418a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } } }
I20260807 18:11:40.101217 31106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:40.100942 31105 sys_catalog.cc:455] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a49b465cdda84e3ab2866ec6fc28418a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a49b465cdda84e3ab2866ec6fc28418a" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34253 } } }
I20260807 18:11:40.101430 31105 sys_catalog.cc:458] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:40.106208 31109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:40.112026 31109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:40.122836 31109 catalog_manager.cc:1347] Generated new cluster ID: 5da59dd22ffd40d492a1c8b9fd0b19c6
I20260807 18:11:40.122970 31109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:40.154626 31109 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:40.156669 31109 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:40.166304 31109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a49b465cdda84e3ab2866ec6fc28418a: Generated new TSK 0
I20260807 18:11:40.167522 31109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:40.179076 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31026
W20260807 18:11:40.180485 31088 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52497
W20260807 18:11:40.181475 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31026: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:40.181648 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31026
W20260807 18:11:40.201278 31011 connection.cc:535] client connection to 127.8.62.190:34253 recv error: Network error: failed to read from TLS socket (remote: 127.8.62.190:34253): Connection reset by peer (error 104)
2026-08-07T18:11:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2023 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:40 dist-test-slave-rnrw krb5kdc[31125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:40 dist-test-slave-rnrw krb5kdc[31125](info): set up 2 sockets
Aug 07 18:11:40 dist-test-slave-rnrw krb5kdc[31125](info): commencing operation
krb5kdc: starting...
W20260807 18:11:41.328117 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.007s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:41 dist-test-slave-rnrw krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:41.634137 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:34737
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:39413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:34737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:41.982720 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:41.983031 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:41.983227 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:41.990762 31138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:41.990887 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:41.990975 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:41.991089 31138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:41.991183 31138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:42.001356 31138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:39413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:34737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:34737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:42.002756 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:42.004168 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:42.016894 31143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:42.016979 31144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:42.017071 31146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:42.018344 31138 server_base.cc:1034] running on GCE node
I20260807 18:11:42.019232 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:42.020823 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:42.022033 31138 hybrid_clock.cc:648] HybridClock initialized: now 1786126302021987 us; error 32 us; skew 500 ppm
I20260807 18:11:42.022432 31138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:42 dist-test-slave-rnrw krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126302, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:42.029438 31138 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:42.031144 31138 webserver.cc:460] Webserver started at http://127.8.62.190:33809/ using document root <none> and password file <none>
I20260807 18:11:42.031880 31138 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:42.032038 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:42.032379 31138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:42.035557 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "a86c3dd4a8a94fd1b970b7098816030f"
format_stamp: "Formatted at 2026-08-07 18:11:42 on dist-test-slave-rnrw"
server_key: "a1acb3ba0b66591b84d60e38bfb367d7"
server_key_iv: "68db40e44cd139e272d91d5e29e71c26"
server_key_version: "encryptionkey@0"
I20260807 18:11:42.036347 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "a86c3dd4a8a94fd1b970b7098816030f"
format_stamp: "Formatted at 2026-08-07 18:11:42 on dist-test-slave-rnrw"
server_key: "a1acb3ba0b66591b84d60e38bfb367d7"
server_key_iv: "68db40e44cd139e272d91d5e29e71c26"
server_key_version: "encryptionkey@0"
I20260807 18:11:42.041797 31138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 18:11:42.045586 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:42.047097 31138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:11:42.047279 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "a86c3dd4a8a94fd1b970b7098816030f"
format_stamp: "Formatted at 2026-08-07 18:11:42 on dist-test-slave-rnrw"
server_key: "a1acb3ba0b66591b84d60e38bfb367d7"
server_key_iv: "68db40e44cd139e272d91d5e29e71c26"
server_key_version: "encryptionkey@0"
I20260807 18:11:42.047508 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:42.053519 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:42.057642 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:42.057936 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:42.077832 31138 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:34737
I20260807 18:11:42.078333 31204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:34737 every 8 connection(s)
I20260807 18:11:42.079802 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:42.083972 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31138
I20260807 18:11:42.084163 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:42.084815 8442 external_mini_cluster.cc:1428] Setting key 8b869990214c7331aefc241295994dfd
I20260807 18:11:42.085151 31205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:42.093154 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:42.086664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47925 (local address 127.8.62.190:34737)
0807 18:11:42.087609 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:42.087623 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:42.087677 (+ 54us) server_negotiation.cc:407] Connection header received
0807 18:11:42.089542 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:42.089565 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:42.089931 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:42.090342 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:42.091450 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:42.091456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:42.091469 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:42.091482 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:42.091565 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:42.091584 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:42.091837 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:42.091993 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 18:11:42.092175 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":365,"threads_started":1}
I20260807 18:11:42.096876 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:42.100484 31205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f: Bootstrap starting.
I20260807 18:11:42.104218 31208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:42.101460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41257 (local address 127.8.62.190:34737)
0807 18:11:42.101694 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:42.101704 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:42.102185 (+ 481us) server_negotiation.cc:407] Connection header received
0807 18:11:42.102384 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:42.102388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:42.102438 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:42.102519 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:42.103510 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:42.103514 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:42.103517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:42.103531 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:42.103569 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:42.103583 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:42.103708 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:42.103875 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 18:11:42.103919 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:11:42.105002 31205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:42.106254 31205 log.cc:824] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:42.109278 31205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f: No bootstrap required, opened a new log
I20260807 18:11:42.117790 31205 raft_consensus.cc:348] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } }
I20260807 18:11:42.118221 31205 raft_consensus.cc:374] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:42.118353 31205 raft_consensus.cc:729] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86c3dd4a8a94fd1b970b7098816030f, State: Initialized, Role: FOLLOWER
I20260807 18:11:42.119005 31205 consensus_queue.cc:260] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } }
I20260807 18:11:42.119251 31205 raft_consensus.cc:388] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:42.119439 31205 raft_consensus.cc:482] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:42.119632 31205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:42.123896 31205 raft_consensus.cc:504] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } }
I20260807 18:11:42.124718 31205 leader_election.cc:302] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a86c3dd4a8a94fd1b970b7098816030f; no voters:
I20260807 18:11:42.125545 31205 leader_election.cc:288] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:42.125720 31215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:42.126574 31215 raft_consensus.cc:686] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [term 1 LEADER]: Becoming Leader. State: Replica: a86c3dd4a8a94fd1b970b7098816030f, State: Running, Role: LEADER
I20260807 18:11:42.127233 31215 consensus_queue.cc:237] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } }
I20260807 18:11:42.128365 31205 sys_catalog.cc:564] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:42.130647 31216 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a86c3dd4a8a94fd1b970b7098816030f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } } }
I20260807 18:11:42.132853 31216 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:42.132082 31217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a86c3dd4a8a94fd1b970b7098816030f. Latest consensus state: current_term: 1 leader_uuid: "a86c3dd4a8a94fd1b970b7098816030f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a86c3dd4a8a94fd1b970b7098816030f" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 34737 } } }
I20260807 18:11:42.134725 31217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:42.136603 31223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:42.145380 31223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:42.155881 31223 catalog_manager.cc:1347] Generated new cluster ID: 7084122b8b0345df83125b03c47c838c
I20260807 18:11:42.156025 31223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:42.178726 31223 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:42.179235 31223 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:42.189574 31223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a86c3dd4a8a94fd1b970b7098816030f: Generated new TSK 0
I20260807 18:11:42.190470 31223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:42.195993 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31138
W20260807 18:11:42.197521 31200 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47925
W20260807 18:11:42.198345 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31138: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:42.198577 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31138
2026-08-07T18:11:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2017 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:42 dist-test-slave-rnrw krb5kdc[31237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:42 dist-test-slave-rnrw krb5kdc[31237](info): set up 2 sockets
Aug 07 18:11:42 dist-test-slave-rnrw krb5kdc[31237](info): commencing operation
krb5kdc: starting...
W20260807 18:11:43.382390 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:43 dist-test-slave-rnrw krb5kdc[31237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:43.689059 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33363
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:34747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:33363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:44.057185 31250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:44.057473 31250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:44.057678 31250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:44.064903 31250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:44.065008 31250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:44.065099 31250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:44.065212 31250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:44.065294 31250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:44.075265 31250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:34747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:33363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:33363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:44.076599 31250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:44.078015 31250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:44.090126 31255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:44.090154 31256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:44.090188 31258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:44.090875 31250 server_base.cc:1034] running on GCE node
I20260807 18:11:44.091671 31250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:44.093349 31250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:44.094574 31250 hybrid_clock.cc:648] HybridClock initialized: now 1786126304094516 us; error 68 us; skew 500 ppm
I20260807 18:11:44.094949 31250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:44 dist-test-slave-rnrw krb5kdc[31237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126304, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:44.098667 31250 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:44.100272 31250 webserver.cc:460] Webserver started at http://127.8.62.190:42247/ using document root <none> and password file <none>
I20260807 18:11:44.101070 31250 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:44.101226 31250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:44.101554 31250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:44.104697 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "46aa5c177a484b30a2d0a7f5741ddbe3"
format_stamp: "Formatted at 2026-08-07 18:11:44 on dist-test-slave-rnrw"
server_key: "8d41115faad7014d21c9f47af845248e"
server_key_iv: "ca3b44f15ea17b8cb7a7014b947e8bb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:44.105520 31250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "46aa5c177a484b30a2d0a7f5741ddbe3"
format_stamp: "Formatted at 2026-08-07 18:11:44 on dist-test-slave-rnrw"
server_key: "8d41115faad7014d21c9f47af845248e"
server_key_iv: "ca3b44f15ea17b8cb7a7014b947e8bb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:44.111217 31250 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:44.115346 31265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:44.117017 31250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:44.117228 31250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "46aa5c177a484b30a2d0a7f5741ddbe3"
format_stamp: "Formatted at 2026-08-07 18:11:44 on dist-test-slave-rnrw"
server_key: "8d41115faad7014d21c9f47af845248e"
server_key_iv: "ca3b44f15ea17b8cb7a7014b947e8bb6"
server_key_version: "encryptionkey@0"
I20260807 18:11:44.117461 31250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:44.141393 31250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:44.146621 31250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:44.146982 31250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:44.168460 31250 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:33363
I20260807 18:11:44.168502 31317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:33363 every 8 connection(s)
I20260807 18:11:44.170019 31250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:44.174199 31318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:44.179716 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31250
I20260807 18:11:44.179955 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:44.180419 8442 external_mini_cluster.cc:1428] Setting key a76b3b7580fd2b670be3de50d26f0ea4
I20260807 18:11:44.186442 31318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3: Bootstrap starting.
I20260807 18:11:44.190119 31318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:44.191237 31318 log.cc:824] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:44.194365 31318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3: No bootstrap required, opened a new log
Aug 07 18:11:44 dist-test-slave-rnrw krb5kdc[31237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:44.203085 31318 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } }
I20260807 18:11:44.203583 31318 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:44.203735 31318 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46aa5c177a484b30a2d0a7f5741ddbe3, State: Initialized, Role: FOLLOWER
I20260807 18:11:44.204524 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:44.182469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59023 (local address 127.8.62.190:33363)
0807 18:11:44.183336 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:44.183350 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:44.183393 (+ 43us) server_negotiation.cc:407] Connection header received
0807 18:11:44.185320 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:44.185348 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:44.185787 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:44.186216 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:44.187745 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:44.189316 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:44.190731 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:44.191506 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:44.191712 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:44.196375 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:44.196383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:44.196403 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:44.196449 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:44.199848 (+ 3399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:44.200676 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:44.200692 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:44.200699 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:44.200895 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:44.201339 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:44.201344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:44.201352 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:44.201944 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:44.202294 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:44.202765 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 18:11:44.203239 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":367,"threads_started":1}
I20260807 18:11:44.204448 31318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } }
I20260807 18:11:44.204802 31318 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:44.204953 31318 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:44.205094 31318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:44.207791 31318 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } }
I20260807 18:11:44.208343 31318 leader_election.cc:302] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46aa5c177a484b30a2d0a7f5741ddbe3; no voters:
I20260807 18:11:44.208760 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:44.209195 31318 leader_election.cc:288] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:44.209304 31324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:44.211884 31318 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:44.212882 31324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [term 1 LEADER]: Becoming Leader. State: Replica: 46aa5c177a484b30a2d0a7f5741ddbe3, State: Running, Role: LEADER
I20260807 18:11:44.213562 31324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } }
I20260807 18:11:44.222981 31325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } } }
I20260807 18:11:44.223495 31325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:44.223354 31326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46aa5c177a484b30a2d0a7f5741ddbe3. Latest consensus state: current_term: 1 leader_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46aa5c177a484b30a2d0a7f5741ddbe3" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 33363 } } }
I20260807 18:11:44.223668 31326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:44.225838 31336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:44.232399 31336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:44.233886 31322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:44.215217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55727 (local address 127.8.62.190:33363)
0807 18:11:44.215407 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:44.215415 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:44.215957 (+ 542us) server_negotiation.cc:407] Connection header received
0807 18:11:44.216200 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:44.216206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:44.216277 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:44.216389 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:44.217936 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:44.218788 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:44.220366 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:44.221076 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:44.221242 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:44.225195 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:44.225201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:44.225205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:44.225247 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:44.229622 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:44.230532 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:44.230537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:44.230540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:44.230725 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:44.231233 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:44.231237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:44.231240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:44.231662 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:44.231903 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:44.233263 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0807 18:11:44.233510 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 18:11:44.242430 31336 catalog_manager.cc:1347] Generated new cluster ID: df45c775b83340c0bcc8b06616eca155
I20260807 18:11:44.242559 31336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:44.256709 31336 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:44.258175 31336 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:44.265867 31336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46aa5c177a484b30a2d0a7f5741ddbe3: Generated new TSK 0
I20260807 18:11:44.266925 31336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:44.273337 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31250
I20260807 18:11:44.274653 31313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:44.367290 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31250
2026-08-07T18:11:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2167 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:44 dist-test-slave-rnrw krb5kdc[31351](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:44 dist-test-slave-rnrw krb5kdc[31351](info): set up 2 sockets
Aug 07 18:11:44 dist-test-slave-rnrw krb5kdc[31351](info): commencing operation
krb5kdc: starting...
W20260807 18:11:45.511701 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:45 dist-test-slave-rnrw krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:45.809155 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:39927
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:38989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:39927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:46.162285 31364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:46.162626 31364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:46.162724 31364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:46.170337 31364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:46.170488 31364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:46.170539 31364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:46.170603 31364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:46.170671 31364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:46.181206 31364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:38989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:39927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:39927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:46.182662 31364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:46.184041 31364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:46.197508 31369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:46.197559 31370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:46.197423 31372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:46.199112 31364 server_base.cc:1034] running on GCE node
I20260807 18:11:46.200021 31364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:46.201695 31364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:46.202965 31364 hybrid_clock.cc:648] HybridClock initialized: now 1786126306202897 us; error 60 us; skew 500 ppm
I20260807 18:11:46.203377 31364 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:46 dist-test-slave-rnrw krb5kdc[31351](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126306, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:46.206956 31364 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:46.208735 31364 webserver.cc:460] Webserver started at http://127.8.62.190:33401/ using document root <none> and password file <none>
I20260807 18:11:46.209473 31364 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:46.209638 31364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:46.209966 31364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:46.213194 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "e4063434c1eb4e56818ce0758217d2a5"
format_stamp: "Formatted at 2026-08-07 18:11:46 on dist-test-slave-rnrw"
server_key: "f4425e0901f8061e0a691e04573bb6b7"
server_key_iv: "da477d419fc44d337ed5677675da627c"
server_key_version: "encryptionkey@0"
I20260807 18:11:46.214002 31364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "e4063434c1eb4e56818ce0758217d2a5"
format_stamp: "Formatted at 2026-08-07 18:11:46 on dist-test-slave-rnrw"
server_key: "f4425e0901f8061e0a691e04573bb6b7"
server_key_iv: "da477d419fc44d337ed5677675da627c"
server_key_version: "encryptionkey@0"
I20260807 18:11:46.219966 31364 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 18:11:46.223693 31379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:46.225168 31364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:46.225373 31364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "e4063434c1eb4e56818ce0758217d2a5"
format_stamp: "Formatted at 2026-08-07 18:11:46 on dist-test-slave-rnrw"
server_key: "f4425e0901f8061e0a691e04573bb6b7"
server_key_iv: "da477d419fc44d337ed5677675da627c"
server_key_version: "encryptionkey@0"
I20260807 18:11:46.225641 31364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:46.248908 31364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:46.253228 31364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:46.253546 31364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:46.273638 31364 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:39927
I20260807 18:11:46.273691 31431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:39927 every 8 connection(s)
I20260807 18:11:46.275095 31364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:46.278177 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31364
I20260807 18:11:46.278376 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:46.278970 8442 external_mini_cluster.cc:1428] Setting key de6874232bd22c342043342e7d119c9d
I20260807 18:11:46.279920 31432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:46.297367 31435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:46.281252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33385 (local address 127.8.62.190:39927)
0807 18:11:46.282314 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:46.282330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:46.282817 (+ 487us) server_negotiation.cc:407] Connection header received
0807 18:11:46.285432 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:46.285466 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:46.286127 (+ 661us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:46.286650 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:46.287263 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:46.289621 (+ 2358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:46.291041 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:46.292249 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:46.292572 (+ 323us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:46.294108 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:46.294119 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:46.294140 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:46.295010 (+ 870us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:46.295123 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:46.295163 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:46.295315 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:46.295536 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 18:11:46.295872 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":780,"thread_start_us":461,"threads_started":1}
I20260807 18:11:46.298242 31432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5: Bootstrap starting.
I20260807 18:11:46.301321 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:46.302155 31432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:46.303571 31432 log.cc:824] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:46.306704 31432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5: No bootstrap required, opened a new log
I20260807 18:11:46.314520 31435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:46.307346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45549 (local address 127.8.62.190:39927)
0807 18:11:46.307617 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:46.307627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:46.308041 (+ 414us) server_negotiation.cc:407] Connection header received
0807 18:11:46.308272 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:46.308276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:46.308340 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:46.308428 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:46.309148 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:46.309898 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:46.311845 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:46.312360 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:46.312459 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:46.313691 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:46.313699 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:46.313702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:46.313723 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:46.313764 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:46.313780 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:46.313856 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:46.314021 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:11:46.314104 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 18:11:46.317035 31432 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } }
I20260807 18:11:46.317494 31432 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:46.317629 31432 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4063434c1eb4e56818ce0758217d2a5, State: Initialized, Role: FOLLOWER
I20260807 18:11:46.318233 31432 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } }
I20260807 18:11:46.318487 31432 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:46.318647 31432 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:46.318837 31432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:46.321287 31432 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } }
I20260807 18:11:46.321841 31432 leader_election.cc:302] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4063434c1eb4e56818ce0758217d2a5; no voters:
I20260807 18:11:46.322469 31432 leader_election.cc:288] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:46.322639 31442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:46.324105 31442 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [term 1 LEADER]: Becoming Leader. State: Replica: e4063434c1eb4e56818ce0758217d2a5, State: Running, Role: LEADER
I20260807 18:11:46.324724 31442 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } }
I20260807 18:11:46.324960 31432 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:46.327989 31443 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4063434c1eb4e56818ce0758217d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } } }
I20260807 18:11:46.328016 31445 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4063434c1eb4e56818ce0758217d2a5. Latest consensus state: current_term: 1 leader_uuid: "e4063434c1eb4e56818ce0758217d2a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4063434c1eb4e56818ce0758217d2a5" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 39927 } } }
I20260807 18:11:46.328455 31443 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:46.328508 31445 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:46.330663 31450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:46.335709 31450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:46.345762 31450 catalog_manager.cc:1347] Generated new cluster ID: 1b7cf921c46f4e61af9cbdb322bedf89
I20260807 18:11:46.345906 31450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:46.364336 31450 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:46.365927 31450 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:46.380606 31450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4063434c1eb4e56818ce0758217d2a5: Generated new TSK 0
I20260807 18:11:46.381531 31450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:46.393128 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31364
W20260807 18:11:46.394446 31427 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33385
W20260807 18:11:46.395251 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31364: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:46.395381 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31364
2026-08-07T18:11:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2028 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:46 dist-test-slave-rnrw krb5kdc[31464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:46 dist-test-slave-rnrw krb5kdc[31464](info): set up 2 sockets
Aug 07 18:11:46 dist-test-slave-rnrw krb5kdc[31464](info): commencing operation
krb5kdc: starting...
W20260807 18:11:47.544096 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:47 dist-test-slave-rnrw krb5kdc[31464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126307, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:47.857420 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37451
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:37451
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:48.214785 31477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:48.215209 31477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:48.215377 31477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:48.223070 31477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:48.223279 31477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:48.223407 31477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:48.223516 31477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:48.223620 31477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:48.234242 31477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:37451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:37451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:48.235684 31477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:48.237207 31477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:48.250514 31482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:48.251550 31485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:48.251799 31483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:48.253062 31477 server_base.cc:1034] running on GCE node
I20260807 18:11:48.254027 31477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:48.255774 31477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:48.257079 31477 hybrid_clock.cc:648] HybridClock initialized: now 1786126308257035 us; error 80 us; skew 500 ppm
I20260807 18:11:48.257526 31477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:48 dist-test-slave-rnrw krb5kdc[31464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126308, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:48.266253 31477 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:48.268196 31477 webserver.cc:460] Webserver started at http://127.8.62.190:43619/ using document root <none> and password file <none>
I20260807 18:11:48.269119 31477 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:48.269287 31477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:48.269637 31477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:48.273025 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "99582134798d4345ab583308699920fb"
format_stamp: "Formatted at 2026-08-07 18:11:48 on dist-test-slave-rnrw"
server_key: "15258fcd210ac2c12e7cb151602ebec1"
server_key_iv: "196b5c633823197e4b9c358c90ba70e8"
server_key_version: "encryptionkey@0"
I20260807 18:11:48.273888 31477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "99582134798d4345ab583308699920fb"
format_stamp: "Formatted at 2026-08-07 18:11:48 on dist-test-slave-rnrw"
server_key: "15258fcd210ac2c12e7cb151602ebec1"
server_key_iv: "196b5c633823197e4b9c358c90ba70e8"
server_key_version: "encryptionkey@0"
I20260807 18:11:48.279639 31477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:48.283591 31492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:48.285156 31477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 18:11:48.285377 31477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "99582134798d4345ab583308699920fb"
format_stamp: "Formatted at 2026-08-07 18:11:48 on dist-test-slave-rnrw"
server_key: "15258fcd210ac2c12e7cb151602ebec1"
server_key_iv: "196b5c633823197e4b9c358c90ba70e8"
server_key_version: "encryptionkey@0"
I20260807 18:11:48.285610 31477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:48.292017 31477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:48.296939 31477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:48.297281 31477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:48.317631 31477 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:37451
I20260807 18:11:48.317727 31543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:37451 every 8 connection(s)
I20260807 18:11:48.319173 31477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:48.323009 31544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:48.328431 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31477
I20260807 18:11:48.328763 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:48.329241 8442 external_mini_cluster.cc:1428] Setting key 3f0fa5e70b20e8eb04569b7b4a0494eb
I20260807 18:11:48.334791 31544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb: Bootstrap starting.
I20260807 18:11:48.338078 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:48.331341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50109 (local address 127.8.62.190:37451)
0807 18:11:48.332384 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:48.332397 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:48.332443 (+ 46us) server_negotiation.cc:407] Connection header received
0807 18:11:48.334309 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:48.334334 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:48.334707 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:48.335145 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:48.336288 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:48.336296 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:48.336313 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:48.336338 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:48.336404 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:48.336432 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:48.336733 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:48.336848 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 18:11:48.337128 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":544,"threads_started":1}
I20260807 18:11:48.338977 31544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:48.340435 31544 log.cc:824] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:48.343037 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:48.344233 31544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb: No bootstrap required, opened a new log
I20260807 18:11:48.350282 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:48.347718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38183 (local address 127.8.62.190:37451)
0807 18:11:48.347925 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:48.347932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:48.348317 (+ 385us) server_negotiation.cc:407] Connection header received
0807 18:11:48.348458 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:48.348463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:48.348533 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:48.348669 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:48.349594 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:48.349600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:48.349605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 18:11:48.349624 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:48.349680 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 18:11:48.349702 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:48.349842 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:48.349900 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 18:11:48.349965 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 18:11:48.353269 31544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } }
I20260807 18:11:48.353654 31544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 18:11:48.353789 31544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99582134798d4345ab583308699920fb, State: Initialized, Role: FOLLOWER
I20260807 18:11:48.354431 31544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } }
I20260807 18:11:48.354676 31544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 18:11:48.354856 31544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 18:11:48.355060 31544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:48.357487 31544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } }
I20260807 18:11:48.357983 31544 leader_election.cc:302] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99582134798d4345ab583308699920fb; no voters:
I20260807 18:11:48.358619 31544 leader_election.cc:288] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 18:11:48.358800 31555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:48.359562 31555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [term 1 LEADER]: Becoming Leader. State: Replica: 99582134798d4345ab583308699920fb, State: Running, Role: LEADER
I20260807 18:11:48.360176 31555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } }
I20260807 18:11:48.361037 31544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:48.363368 31556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99582134798d4345ab583308699920fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } } }
I20260807 18:11:48.363722 31556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:48.364526 31557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99582134798d4345ab583308699920fb. Latest consensus state: current_term: 1 leader_uuid: "99582134798d4345ab583308699920fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99582134798d4345ab583308699920fb" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 37451 } } }
I20260807 18:11:48.364861 31557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:48.365831 31560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:48.371320 31560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:48.385699 31560 catalog_manager.cc:1347] Generated new cluster ID: 4e9d20852037451a91a83f534831aae8
I20260807 18:11:48.385840 31560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:48.416934 31560 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:48.417464 31560 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:48.424934 31560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99582134798d4345ab583308699920fb: Generated new TSK 0
I20260807 18:11:48.425911 31560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:48.435250 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31477
W20260807 18:11:48.436604 31539 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50109
W20260807 18:11:48.437453 8442 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31477: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 18:11:48.437651 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31477
2026-08-07T18:11:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2043 ms)
[----------] 18 tests from AuthTokenIssuingTest (37177 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:48 dist-test-slave-rnrw krb5kdc[31577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:48 dist-test-slave-rnrw krb5kdc[31577](info): set up 2 sockets
Aug 07 18:11:48 dist-test-slave-rnrw krb5kdc[31577](info): commencing operation
krb5kdc: starting...
W20260807 18:11:49.593966 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:49 dist-test-slave-rnrw krb5kdc[31577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:49.909431 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35605
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:50.257339 31591 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:50.257686 31591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:50.257764 31591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:50.257850 31591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:50.265298 31591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:50.265413 31591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:50.265475 31591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:50.265520 31591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:50.265594 31591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:50.275969 31591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:50.277427 31591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:50.278849 31591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:50.292497 31596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:50.292527 31597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:50.292598 31599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:50.293350 31591 server_base.cc:1034] running on GCE node
I20260807 18:11:50.294643 31591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:50.296262 31591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:50.297595 31591 hybrid_clock.cc:648] HybridClock initialized: now 1786126310297488 us; error 79 us; skew 500 ppm
I20260807 18:11:50.297994 31591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:50 dist-test-slave-rnrw krb5kdc[31577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:50.301733 31591 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:50.303552 31591 webserver.cc:460] Webserver started at http://127.8.62.190:41701/ using document root <none> and password file <none>
I20260807 18:11:50.304347 31591 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:50.304497 31591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:50.304905 31591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:50.308447 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "7c2cdea99f534bcb8bcbde92db227162"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "a7095140b2a9df2303fd1502dd604964"
server_key_iv: "725b093fe1ddd4f9394636372a558d66"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.309485 31591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "7c2cdea99f534bcb8bcbde92db227162"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "a7095140b2a9df2303fd1502dd604964"
server_key_iv: "725b093fe1ddd4f9394636372a558d66"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.315335 31591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:50.319407 31606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:50.321237 31591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:11:50.321471 31591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "7c2cdea99f534bcb8bcbde92db227162"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "a7095140b2a9df2303fd1502dd604964"
server_key_iv: "725b093fe1ddd4f9394636372a558d66"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.321738 31591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:50.344748 31591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:50.349782 31591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:50.350106 31591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:50.370265 31591 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:35605
I20260807 18:11:50.370277 31657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:35605 every 8 connection(s)
I20260807 18:11:50.371711 31591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:50.375878 31658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:50.379679 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31591
I20260807 18:11:50.379918 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:50.380815 8442 external_mini_cluster.cc:1428] Setting key 8d237b6a9883f50929d73f28f74a634e
I20260807 18:11:50.385663 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:50 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.189@KRBTEST.COM; defaulting to no policy
I20260807 18:11:50.416198 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.390710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35605 (local address 127.0.0.1:40184)
0807 18:11:50.393184 (+ 2474us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.393237 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:50.395333 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:50.396917 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:50.396929 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:50.397872 (+ 943us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:50.398440 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.398477 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.400219 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.400227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.402315 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.402334 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.403407 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.403417 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.403657 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.404418 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:50.404453 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:50.407400 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:50.411067 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.411082 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.411097 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.411533 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.412011 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.412017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.412021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.412210 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.413180 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:50.413193 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:50.413450 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 18:11:50.413756 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 18:11:50.414467 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2170,"mutex_wait_us":171,"thread_start_us":1921,"threads_started":1}
I20260807 18:11:50.416164 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.390710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.8.62.190:35605)
0807 18:11:50.396086 (+ 5376us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:50.396092 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:50.396125 (+ 33us) server_negotiation.cc:407] Connection header received
0807 18:11:50.396277 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:50.396283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:50.396616 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:50.396823 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:50.398730 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.400018 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.402491 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.403106 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.403263 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.407703 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:50.407714 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:50.407728 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:50.407768 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:50.410893 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.411708 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.411715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.411717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.411882 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.412337 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.412342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.412347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.412987 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:50.413263 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:50.413904 (+ 641us) server_negotiation.cc:299] Negotiation successful
0807 18:11:50.414339 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":5173,"thread_start_us":4938,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.189@KRBTEST.COM" created.
I20260807 18:11:50.420089 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:50.421422 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.420879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58876)
0807 18:11:50.421103 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.421244 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 18:11:50.421736 31609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
I20260807 18:11:50.423756 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.423235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58886)
0807 18:11:50.423442 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.423550 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:11:50.424057 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:45895: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
I20260807 18:11:50.454443 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:45895
--webserver_interface=127.8.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:11:50.473486 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } attempt: 1
I20260807 18:11:50.474812 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.474328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58888)
0807 18:11:50.474558 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.474669 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:11:50.475930 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.475570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58890)
0807 18:11:50.475727 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.475799 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:11:50.476114 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:45895: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
I20260807 18:11:50.544402 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } attempt: 2
I20260807 18:11:50.545980 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.545316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58896)
0807 18:11:50.545631 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.545725 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 18:11:50.547299 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.546790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58902)
0807 18:11:50.546966 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.547092 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 18:11:50.547595 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:45895: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
I20260807 18:11:50.638953 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } attempt: 3
I20260807 18:11:50.640448 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.639938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58910)
0807 18:11:50.640195 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.640298 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260807 18:11:50.641776 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.641365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58912)
0807 18:11:50.641535 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.641619 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260807 18:11:50.642071 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:45895: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
I20260807 18:11:50.785346 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } attempt: 4
I20260807 18:11:50.786761 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.786145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58922)
0807 18:11:50.786472 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.786624 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260807 18:11:50.787861 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.787489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58934)
0807 18:11:50.787642 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.787740 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:11:50.788059 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:45895: Network error: Client connection negotiation failed: client connection to 127.8.62.189:45895: connect: Connection refused (error 111)
W20260807 18:11:50.810830 31665 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:50.811270 31665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:50.811409 31665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:50.811555 31665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:50.819444 31665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:50.819618 31665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:50.819747 31665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:50.819859 31665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:50.819955 31665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:50.830575 31665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:45895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:50.831987 31665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:50.833376 31665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:50.845913 31671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:50.845861 31674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:50.845817 31672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:50.846822 31665 server_base.cc:1034] running on GCE node
I20260807 18:11:50.847694 31665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:50.849380 31665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:50.850682 31665 hybrid_clock.cc:648] HybridClock initialized: now 1786126310850579 us; error 58 us; skew 500 ppm
I20260807 18:11:50.851044 31665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:50 dist-test-slave-rnrw krb5kdc[31577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:50.854568 31665 init.cc:377] Logged in from keytab as kudu/127.8.62.189@KRBTEST.COM (short username kudu)
I20260807 18:11:50.856241 31665 webserver.cc:460] Webserver started at http://127.8.62.189:43587/ using document root <none> and password file <none>
I20260807 18:11:50.857056 31665 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:50.857156 31665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:50.857430 31665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:50.860670 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data/instance:
uuid: "a451ab5d90cc4835879833f5aafdd191"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "b86a25014661c2b294989de8c1095160"
server_key_iv: "70cbc86190d52487defd6917a92a17dc"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.861491 31665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal/instance:
uuid: "a451ab5d90cc4835879833f5aafdd191"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "b86a25014661c2b294989de8c1095160"
server_key_iv: "70cbc86190d52487defd6917a92a17dc"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.867333 31665 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 18:11:50.871466 31681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:50.873311 31665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 18:11:50.873520 31665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "a451ab5d90cc4835879833f5aafdd191"
format_stamp: "Formatted at 2026-08-07 18:11:50 on dist-test-slave-rnrw"
server_key: "b86a25014661c2b294989de8c1095160"
server_key_iv: "70cbc86190d52487defd6917a92a17dc"
server_key_version: "encryptionkey@0"
I20260807 18:11:50.873805 31665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:50.906335 31665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:50.911070 31665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:50.911401 31665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:50.931982 31665 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.189:45895
I20260807 18:11:50.932030 31734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.189:45895 every 8 connection(s)
I20260807 18:11:50.933477 31665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
I20260807 18:11:50.935180 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31665
I20260807 18:11:50.935366 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-1/wal/instance
I20260807 18:11:50.935879 8442 external_mini_cluster.cc:1428] Setting key 92400f2b6c4be898beb2b7c2eb237b4a
I20260807 18:11:50.938380 31735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:50.948995 31735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:50 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.188@KRBTEST.COM" created.
I20260807 18:11:50.973038 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.951866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.8.62.190:35605)
0807 18:11:50.952456 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:50.952465 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:50.953530 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 18:11:50.955462 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:50.955471 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:50.955571 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:50.955719 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:50.957567 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.958857 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.960631 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.961416 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.961576 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.965338 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:50.965349 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:50.965357 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:50.965417 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:50.969093 (+ 3676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.970183 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.970192 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.970199 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.970370 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.970962 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.970966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.970968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.971277 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:50.971398 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:50.972426 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0807 18:11:50.972755 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":285,"threads_started":1}
I20260807 18:11:50.974426 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.952431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35605 (local address 127.0.0.1:40188)
0807 18:11:50.953328 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.953381 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:50.955171 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:50.955976 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:50.955993 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:50.956719 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:50.957313 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.957331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.959057 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.959061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.960371 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.960390 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.961680 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.961686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.961913 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.962705 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:50.962728 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:50.965079 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:50.969285 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.969296 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.969316 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.970022 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.970513 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.970521 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.970527 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.970811 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.971436 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:50.971447 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:50.971987 (+ 540us) client_negotiation.cc:769] Sending connection context
0807 18:11:50.972311 (+ 324us) client_negotiation.cc:240] Negotiation successful
0807 18:11:50.972930 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":364,"threads_started":1}
I20260807 18:11:50.976464 31735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:50 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:50.994620 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.977206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58950)
0807 18:11:50.977475 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.977501 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:50.977614 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:50.978742 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:50.978749 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:50.979364 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:50.979590 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.979605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.981858 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.981867 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.983522 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:50.983541 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.984440 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.984449 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:50.984707 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.985330 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:50.985359 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:50.987948 (+ 2589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:50.991795 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.991804 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.991808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.992208 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.992687 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:50.992693 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:50.992697 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:50.992839 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:50.993523 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:50.993527 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:50.993691 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 18:11:50.993851 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 18:11:50.994179 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 18:11:50.994746 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.977511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.8.62.189:45895)
0807 18:11:50.978273 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:50.978282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:50.978314 (+ 32us) server_negotiation.cc:407] Connection header received
0807 18:11:50.978358 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:50.978364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:50.978555 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:50.978675 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:50.979862 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.981689 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.983690 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:50.984259 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:50.984359 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:50.988224 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:50.988237 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:50.988250 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:50.988278 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:50.991626 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.992373 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.992385 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.992388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.992528 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:50.992957 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:50.992965 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:50.992971 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:50.993359 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:50.993605 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:50.994020 (+ 415us) server_negotiation.cc:299] Negotiation successful
0807 18:11:50.994312 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":578,"thread_start_us":431,"threads_started":1}
I20260807 18:11:50.998049 31735 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:50.999439 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:50.998922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56902)
0807 18:11:50.999133 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:50.999254 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 18:11:50.999830 31684 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.001646 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.001247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56906)
0807 18:11:51.001430 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.001534 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:11:51.001861 31735 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
I20260807 18:11:51.009657 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:34553
--webserver_interface=127.8.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:11:51.051250 31735 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 1
I20260807 18:11:51.052503 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.052019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56916)
0807 18:11:51.052234 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.052351 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:11:51.053617 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.053266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56932)
0807 18:11:51.053419 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.053515 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:11:51.053830 31735 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.087338 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } attempt: 5
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:51.103859 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.088187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58952)
0807 18:11:51.088430 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.088453 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.088621 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.089017 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.089025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.089621 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.089864 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.089877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.091131 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.091138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.092619 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.092636 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.093527 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.093533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.093755 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.094450 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.094474 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.097541 (+ 3067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.100916 (+ 3375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.100922 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.100926 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.101466 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.101909 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.101914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.101918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.102106 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.102791 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.102797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.102978 (+ 181us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.103168 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.103490 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 18:11:51.103863 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.088469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58952 (local address 127.8.62.189:45895)
0807 18:11:51.088696 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.088701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.088720 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:11:51.088774 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.088778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.088839 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.088924 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.090093 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.090925 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.092799 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.093345 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.093455 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.097808 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.097814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.097818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.097856 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.100761 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.101610 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.101614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.101617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.101783 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.102230 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.102261 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.102266 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.102604 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.102814 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.103289 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.103516 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:11:51.105181 31658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:51.106221 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.105808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56948)
0807 18:11:51.105983 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.106073 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 18:11:51.107184 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.106856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56960)
0807 18:11:51.106999 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.107074 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 18:11:51.107451 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.122076 31735 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 2
I20260807 18:11:51.123297 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.122840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56976)
0807 18:11:51.123035 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.123137 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:11:51.124336 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.123994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:56986)
0807 18:11:51.124142 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.124222 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:11:51.124576 31735 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.158743 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 1
I20260807 18:11:51.160140 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.159491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57000)
0807 18:11:51.159786 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.159905 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 18:11:51.161579 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.161108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57014)
0807 18:11:51.161302 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.161404 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260807 18:11:51.161880 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.215831 31735 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 3
I20260807 18:11:51.217311 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.216665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57026)
0807 18:11:51.216982 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.217101 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:11:51.218732 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.218154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57038)
0807 18:11:51.218471 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.218574 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260807 18:11:51.219048 31735 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.230149 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 2
I20260807 18:11:51.231448 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.230854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57046)
0807 18:11:51.231120 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.231241 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 18:11:51.232595 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.232171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57058)
0807 18:11:51.232353 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.232441 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 18:11:51.232851 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.339121 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 3
I20260807 18:11:51.340466 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.339827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57064)
0807 18:11:51.340110 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.340233 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 18:11:51.341665 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.341234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57072)
0807 18:11:51.341428 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.341528 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 18:11:51.341922 31658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
I20260807 18:11:51.362258 31735 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 4
I20260807 18:11:51.363521 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.362932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57074)
0807 18:11:51.363204 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.363316 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 18:11:51.364667 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.364233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57082)
0807 18:11:51.364406 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.364498 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 18:11:51.364885 31735 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:34553: Network error: Client connection negotiation failed: client connection to 127.8.62.188:34553: connect: Connection refused (error 111)
W20260807 18:11:51.370556 31743 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:51.370965 31743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:51.371086 31743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:51.371222 31743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:51.379212 31743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:51.379304 31743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:51.379396 31743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:51.379477 31743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:51.379561 31743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:51.390185 31743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:35605,127.8.62.189:45895,127.8.62.188:34553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:51.391592 31743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:51.393088 31743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:51.405581 31748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:51.405486 31751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:51.405522 31749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:51.406442 31743 server_base.cc:1034] running on GCE node
I20260807 18:11:51.407196 31743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:51.408936 31743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:51.410169 31743 hybrid_clock.cc:648] HybridClock initialized: now 1786126311410102 us; error 37 us; skew 500 ppm
I20260807 18:11:51.410547 31743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126311, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:51.413827 31743 init.cc:377] Logged in from keytab as kudu/127.8.62.188@KRBTEST.COM (short username kudu)
I20260807 18:11:51.415427 31743 webserver.cc:460] Webserver started at http://127.8.62.188:44427/ using document root <none> and password file <none>
I20260807 18:11:51.416215 31743 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:51.416359 31743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:51.416778 31743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:51.420110 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data/instance:
uuid: "86b1d510d95142b1b23bb581b33d9849"
format_stamp: "Formatted at 2026-08-07 18:11:51 on dist-test-slave-rnrw"
server_key: "7db8dbd8b37670437d8f77530b2e4bf2"
server_key_iv: "1126d6992d1090fd10efa33ea72257fb"
server_key_version: "encryptionkey@0"
I20260807 18:11:51.421008 31743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal/instance:
uuid: "86b1d510d95142b1b23bb581b33d9849"
format_stamp: "Formatted at 2026-08-07 18:11:51 on dist-test-slave-rnrw"
server_key: "7db8dbd8b37670437d8f77530b2e4bf2"
server_key_iv: "1126d6992d1090fd10efa33ea72257fb"
server_key_version: "encryptionkey@0"
I20260807 18:11:51.426926 31743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260807 18:11:51.430982 31758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:51.432734 31743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 18:11:51.432951 31743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "86b1d510d95142b1b23bb581b33d9849"
format_stamp: "Formatted at 2026-08-07 18:11:51 on dist-test-slave-rnrw"
server_key: "7db8dbd8b37670437d8f77530b2e4bf2"
server_key_iv: "1126d6992d1090fd10efa33ea72257fb"
server_key_version: "encryptionkey@0"
I20260807 18:11:51.433210 31743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:51.452363 31743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:51.457135 31743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:51.457437 31743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:51.477965 31743 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.188:34553
I20260807 18:11:51.477988 31809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.188:34553 every 8 connection(s)
I20260807 18:11:51.479494 31743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
I20260807 18:11:51.479961 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 31743
I20260807 18:11:51.480158 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786125932226923-8442-0/minicluster-data/master-2/wal/instance
I20260807 18:11:51.480854 8442 external_mini_cluster.cc:1428] Setting key 5792f1f2995c5a6957a55d79210461d8
I20260807 18:11:51.484890 31810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:51.494757 31810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:11:51.507210 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.482927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57086 (local address 127.8.62.188:34553)
0807 18:11:51.483785 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.483802 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.483874 (+ 72us) server_negotiation.cc:407] Connection header received
0807 18:11:51.486486 (+ 2612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.486517 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.487246 (+ 729us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.487785 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.489394 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.491631 (+ 2237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.493049 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.494122 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.494305 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.498974 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.498984 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.498998 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.499034 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.502691 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.503540 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.503547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.503589 (+ 42us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.503760 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.504180 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.504186 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.504189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.504897 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.505132 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.505692 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.506110 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":334,"threads_started":1}
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126311, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:51.514755 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.497681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:35605 (local address 127.0.0.1:40198)
0807 18:11:51.498731 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.498781 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.498967 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.499373 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.499381 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.500255 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.500650 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.500685 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.502230 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.502234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.503722 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.503734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.504719 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.504724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.504888 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.505673 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.505700 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.508017 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.511990 (+ 3973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.511999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.512004 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.512411 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.512884 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.512890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.512893 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.513151 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.513747 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.513754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.513949 (+ 195us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.514183 (+ 234us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.514444 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"spinlock_wait_cycles":20352,"thread_start_us":591,"threads_started":1}
I20260807 18:11:51.514963 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.497727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.8.62.190:35605)
0807 18:11:51.498346 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.498352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.498889 (+ 537us) server_negotiation.cc:407] Connection header received
0807 18:11:51.499115 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.499121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.499205 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.499330 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.500863 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.502031 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.503887 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.504462 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.504628 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.508231 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.508238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.508243 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.508287 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.511855 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.512568 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.512577 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.512583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.512759 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.513273 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.513280 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.513282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.513595 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.513717 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.514326 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.514557 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":457,"thread_start_us":294,"threads_started":1}
I20260807 18:11:51.516398 31810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:51.519167 31658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 4
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126311, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:11:51.529052 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.510404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.8.62.190:35605)
0807 18:11:51.511123 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.511127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.511147 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:11:51.511187 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.511190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.511246 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.511333 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.512917 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.514100 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.515650 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.516126 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.516272 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.520982 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.520990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.520995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.521035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.525177 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.526099 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.526107 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.526113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.526300 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.526708 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.526714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.526717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.527132 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.527445 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.528058 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.528313 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":597,"thread_start_us":326,"threads_started":1}
I20260807 18:11:51.534158 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.517355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.8.62.189:45895)
0807 18:11:51.517615 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.517622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.517646 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:11:51.517694 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.517698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.517755 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.517872 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.518585 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.519410 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.521197 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.521715 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.521878 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.526278 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.526287 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.526293 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.526339 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.530254 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.531138 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.531143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.531147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.531349 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.531874 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.531879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.531882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.532344 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.532500 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.533174 (+ 674us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.533374 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":57}
I20260807 18:11:51.535351 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.517199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:45895 (local address 127.0.0.1:58960)
0807 18:11:51.517381 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.517402 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.517510 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.517895 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.517899 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.518282 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.518422 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.518431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.519574 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.519583 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.521026 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.521041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.521912 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.521918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.522113 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.523293 (+ 1180us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.523320 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.526054 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.530402 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.530407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.530410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.531027 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.531498 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.531502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.531505 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.531728 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.532623 (+ 895us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.532628 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.532851 (+ 223us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.534421 (+ 1570us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.534895 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 18:11:51.535997 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.519729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57100)
0807 18:11:51.519980 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.519998 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.520131 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.520506 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.520511 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.521012 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.521269 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.521284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.522647 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.522651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.524022 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.524037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.524974 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.524980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.525227 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.525777 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.525804 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.528946 (+ 3142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.533168 (+ 4222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.533174 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.533178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.533673 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.534111 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.534117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.534120 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.534328 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.534926 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.534931 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.535104 (+ 173us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.535282 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.535607 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 18:11:51.536293 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.519794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.8.62.188:34553)
0807 18:11:51.519956 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.519961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.520084 (+ 123us) server_negotiation.cc:407] Connection header received
0807 18:11:51.520251 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.520257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.520333 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.520459 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.521428 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.522509 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.524204 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.524816 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.524967 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.529253 (+ 4286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.529260 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.529264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.529301 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.533021 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.533801 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.533809 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.533813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.533993 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.534481 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.534486 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.534489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.534788 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.534960 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.535414 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.535962 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:11:51.536392 31810 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:51.543026 31658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: Bootstrap starting.
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126311, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:11:51.547499 31658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:51.549110 31658 log.cc:824] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:51.550134 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.531074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.8.62.189:45895)
0807 18:11:51.531729 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.531735 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.531760 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:11:51.531813 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.531819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.531889 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.532017 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.533543 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.535070 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.537038 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.537641 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.537776 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.543169 (+ 5393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.543179 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.543184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.543231 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.546401 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.547148 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.547154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.547158 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.547357 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.547912 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.547920 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.547927 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.548328 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.548973 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.549402 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.549677 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":266,"threads_started":1}
I20260807 18:11:51.553618 31815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.537115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57114)
0807 18:11:51.537495 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.537519 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.537618 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.538162 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.538168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.538668 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.538861 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.538876 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.540258 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.540265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.541563 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.541576 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.542407 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.542413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.542630 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.543900 (+ 1270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.543926 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.546436 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.550610 (+ 4174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.550615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.550619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.551031 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.551581 (+ 550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.551589 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.551595 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.551775 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.552437 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.552443 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.552650 (+ 207us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.552849 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.553213 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260807 18:11:51.553901 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.537648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57114 (local address 127.8.62.188:34553)
0807 18:11:51.537874 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.537881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.537895 (+ 14us) server_negotiation.cc:407] Connection header received
0807 18:11:51.537938 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.537943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.538012 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.538109 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.539030 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.540096 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.541720 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.542242 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.542388 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.546666 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.546673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.546677 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.546711 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.550457 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.551184 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.551194 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.551200 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.551406 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.551913 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.551919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.551922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.552296 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.552479 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.552997 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.553496 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":250,"server-negotiator.queue_time_us":63}
I20260807 18:11:51.556380 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:51.558568 31810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: Bootstrap starting.
I20260807 18:11:51.561524 31658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: No bootstrap required, opened a new log
I20260807 18:11:51.559350 31633 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7c2cdea99f534bcb8bcbde92db227162"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40204
I20260807 18:11:51.563730 31810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:51.565128 31810 log.cc:824] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:51.568863 31810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: No bootstrap required, opened a new log
I20260807 18:11:51.569485 31658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.569921 31658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c2cdea99f534bcb8bcbde92db227162, State: Initialized, Role: FOLLOWER
I20260807 18:11:51.570473 31658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.571509 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.571914 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.572847 31658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:51.572964 31633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:11:51.573197 31633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:51.575826 31633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.577222 31633 leader_election.cc:288] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [CANDIDATE]: Term 1 election: Requested vote from peers a451ab5d90cc4835879833f5aafdd191 (127.8.62.189:45895), 86b1d510d95142b1b23bb581b33d9849 (127.8.62.188:34553)
I20260807 18:11:51.584241 31810 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.584892 31810 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b1d510d95142b1b23bb581b33d9849, State: Initialized, Role: FOLLOWER
I20260807 18:11:51.585577 31810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.589074 31810 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:51.589154 31785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c2cdea99f534bcb8bcbde92db227162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "86b1d510d95142b1b23bb581b33d9849"
I20260807 18:11:51.589835 31785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:51.594734 31785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2cdea99f534bcb8bcbde92db227162 in term 1.
I20260807 18:11:51.595844 31609 leader_election.cc:302] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c2cdea99f534bcb8bcbde92db227162, 86b1d510d95142b1b23bb581b33d9849; no voters:
I20260807 18:11:51.596639 31820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:51.597472 31820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [term 1 LEADER]: Becoming Leader. State: Replica: 7c2cdea99f534bcb8bcbde92db227162, State: Running, Role: LEADER
I20260807 18:11:51.598207 31820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
W20260807 18:11:51.603185 31612 tablet.cc:2369] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:51.587664 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.606237 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.613948 31832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2cdea99f534bcb8bcbde92db227162. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.615557 31832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:51.623546 31841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:51.638397 31852 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:11:51.649282 31816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.604784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40214 (local address 127.8.62.190:35605)
0807 18:11:51.605077 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.605087 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.618942 (+ 13855us) server_negotiation.cc:407] Connection header received
0807 18:11:51.619769 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.619778 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.619885 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.619975 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.626370 (+ 6395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.627810 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.631861 (+ 4051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.632763 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.632888 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.642042 (+ 9154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.642050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.642055 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.642102 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.645934 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.646825 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.646831 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.646835 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.647005 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.647494 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.647500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.647504 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.647898 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.648180 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.648634 (+ 454us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.648881 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 18:11:51.651599 31841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:51.641782 31743 master_runner.cc:186] Error getting master registration for 127.8.62.190:35605: OK, instance_id { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" instance_seqno: 1786126310364632 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:11:51.653625 31743 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:11:51.663313 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.615226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.8.62.189:45895)
0807 18:11:51.615442 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.615448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.634773 (+ 19325us) server_negotiation.cc:407] Connection header received
0807 18:11:51.638119 (+ 3346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.638129 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.638285 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.638375 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.640639 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.642106 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.645619 (+ 3513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.646321 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.646452 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.656184 (+ 9732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.656196 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.656203 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.656264 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.659913 (+ 3649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.660840 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.660846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.660849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.661007 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.661490 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.661495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.661499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.661867 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.662119 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.662694 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.662953 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 18:11:51.664072 31735 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } attempt: 5
I20260807 18:11:51.667273 31813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.635319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.8.62.188:34553)
0807 18:11:51.635562 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.635571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.649228 (+ 13657us) server_negotiation.cc:407] Connection header received
0807 18:11:51.649885 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.649892 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.649985 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.650106 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.651646 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.652545 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.654348 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.655007 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.655163 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.658441 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.658450 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.658458 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.658505 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.662890 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.664203 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.664211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.664216 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.664367 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.664868 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.664875 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.664880 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.665257 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.665526 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.666601 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.666858 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 18:11:51 dist-test-slave-rnrw krb5kdc[31577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126310, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:11:51.660991 31591 master_runner.cc:186] Error getting master registration for 127.8.62.189:45895: OK, instance_id { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" instance_seqno: 1786126310926331 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:11:51.676249 31591 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:11:51.681887 31785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [term 1 FOLLOWER]: Refusing update from remote peer 7c2cdea99f534bcb8bcbde92db227162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:51.682941 31832 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:51.683676 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.664614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:34553 (local address 127.0.0.1:57142)
0807 18:11:51.665141 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:51.665160 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:51.665259 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:51.665791 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:51.665796 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:51.666283 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:51.666495 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.666508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.668722 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.668728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.670128 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:51.670145 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.671190 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.671196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:51.671418 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.672103 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:51.672128 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:51.675874 (+ 3746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:51.680432 (+ 4558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.680438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.680442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.681128 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.681642 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:51.681647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:51.681651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:51.681820 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:51.682590 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:51.682595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:51.682767 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 18:11:51.682910 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 18:11:51.683270 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400}
I20260807 18:11:51.685019 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.664689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57142 (local address 127.8.62.188:34553)
0807 18:11:51.665461 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.665467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.665491 (+ 24us) server_negotiation.cc:407] Connection header received
0807 18:11:51.665542 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.665548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.665618 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.666648 (+ 1030us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.666909 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.668524 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.670338 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.670983 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.671762 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.676181 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:51.676188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:51.676193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:51.676234 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:51.680247 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.681325 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.681330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.681334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.681491 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:51.681978 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:51.681983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:51.682035 (+ 52us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:51.682439 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:51.684185 (+ 1746us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.684316 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.684591 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":626,"threads_started":1}
I20260807 18:11:51.692243 31735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: Bootstrap starting.
I20260807 18:11:51.698176 31735 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:51.698908 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2cdea99f534bcb8bcbde92db227162. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.699231 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.699640 31735 log.cc:824] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:51.702524 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.702775 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.704288 31857 mvcc.cc:205] Tried to move back new op lower bound from 7315973372637175808 to 7315973372312907776. Current Snapshot: MvccSnapshot[applied={T|T < 7315973372637175808}]
I20260807 18:11:51.712751 31841 catalog_manager.cc:1347] Generated new cluster ID: 55a5f18614224151987a5ffb036a8d4d
I20260807 18:11:51.712882 31841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:51.713126 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.713486 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:51.717655 31735 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: No bootstrap required, opened a new log
I20260807 18:11:51.729616 31735 raft_consensus.cc:348] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.730087 31735 raft_consensus.cc:729] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a451ab5d90cc4835879833f5aafdd191, State: Initialized, Role: FOLLOWER
I20260807 18:11:51.730635 31735 consensus_queue.cc:260] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } }
I20260807 18:11:51.734303 31841 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:51.736217 31841 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:51.745172 31864 sys_catalog.cc:455] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.745661 31864 sys_catalog.cc:458] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.746768 31735 sys_catalog.cc:564] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:51.747180 31708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:51.748493 31709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7c2cdea99f534bcb8bcbde92db227162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a451ab5d90cc4835879833f5aafdd191"
I20260807 18:11:51.752048 31708 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [term 1 FOLLOWER]: Refusing update from remote peer 7c2cdea99f534bcb8bcbde92db227162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:51.757293 31841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162: Generated new TSK 0
I20260807 18:11:51.758441 31841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:51.761309 31820 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:51.762506 31709 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c2cdea99f534bcb8bcbde92db227162 in term 1.
I20260807 18:11:51.789644 31864 sys_catalog.cc:455] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c2cdea99f534bcb8bcbde92db227162. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.789988 31864 sys_catalog.cc:458] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.790452 31875 sys_catalog.cc:455] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:51.790778 31875 sys_catalog.cc:458] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:51.822808 31881 catalog_manager.cc:1259] Loaded cluster ID: 55a5f18614224151987a5ffb036a8d4d
I20260807 18:11:51.823163 31881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: loading cluster ID for follower catalog manager: success
I20260807 18:11:51.827342 31881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: acquiring CA information for follower catalog manager: success
I20260807 18:11:51.830217 31881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a451ab5d90cc4835879833f5aafdd191: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:11:51.859763 31816 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.856844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.8.62.190:35605)
0807 18:11:51.857062 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.857069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.857614 (+ 545us) server_negotiation.cc:407] Connection header received
0807 18:11:51.857919 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.857926 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.858014 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.858107 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.859422 (+ 1315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40224: BlockingRecv error: recv got EOF from 127.0.0.1:40224 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260807 18:11:51.862177 31817 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.857947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58996 (local address 127.8.62.189:45895)
0807 18:11:51.858127 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.858139 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.860405 (+ 2266us) server_negotiation.cc:407] Connection header received
0807 18:11:51.860709 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.860715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.860817 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.860890 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.861873 (+ 983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58996: BlockingRecv error: recv got EOF from 127.0.0.1:58996 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:11:51.865773 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.863574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.8.62.188:34553)
0807 18:11:51.863761 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.863767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.863946 (+ 179us) server_negotiation.cc:407] Connection header received
0807 18:11:51.864130 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.864139 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.864236 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.864342 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.865539 (+ 1197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57150: BlockingRecv error: recv got EOF from 127.0.0.1:57150 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:11:51.878755 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.870262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58998 (local address 127.8.62.189:45895)
0807 18:11:51.870430 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.870435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.870915 (+ 480us) server_negotiation.cc:407] Connection header received
0807 18:11:51.871175 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.871181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.871405 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.871586 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:51.872138 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.873453 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.875562 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.876288 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.876420 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.877068 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:51.877938 (+ 870us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:51.878110 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.878220 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.878312 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 18:11:51.889097 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.887088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57152 (local address 127.8.62.188:34553)
0807 18:11:51.887273 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.887280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.887354 (+ 74us) server_negotiation.cc:407] Connection header received
0807 18:11:51.887712 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.887719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.887852 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.887963 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.888827 (+ 864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57152: BlockingRecv error: recv got EOF from 127.0.0.1:57152 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 18:11:51.903571 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.894893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.8.62.189:45895)
0807 18:11:51.895091 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.895097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.897148 (+ 2051us) server_negotiation.cc:407] Connection header received
0807 18:11:51.897332 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.897339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.897437 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.897540 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:51.898060 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.899236 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.901293 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.902026 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.902170 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.902681 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:51.902842 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:51.902938 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.903134 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.903225 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 18:11:51.914096 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.912173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57158 (local address 127.8.62.188:34553)
0807 18:11:51.912413 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.912421 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.912443 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:11:51.912518 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.912524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.912653 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.912759 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.913808 (+ 1049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57158: BlockingRecv error: recv got EOF from 127.0.0.1:57158 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:11:51.934204 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.927002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59020 (local address 127.8.62.189:45895)
0807 18:11:51.927210 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.927218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.927241 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:11:51.927490 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.927497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.927600 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.927669 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:51.928390 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.929613 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.931730 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.932475 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.932629 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.933274 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:51.933469 (+ 195us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:51.933540 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.933788 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.933883 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:11:51.972455 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:51.969744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57172 (local address 127.8.62.188:34553)
0807 18:11:51.969939 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.969945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.970498 (+ 553us) server_negotiation.cc:407] Connection header received
0807 18:11:51.970911 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.970920 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.971029 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.971137 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:51.972223 (+ 1086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57172: BlockingRecv error: recv got EOF from 127.0.0.1:57172 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 18:11:51.989329 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:51.981951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.8.62.189:45895)
0807 18:11:51.982217 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:51.982223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:51.982330 (+ 107us) server_negotiation.cc:407] Connection header received
0807 18:11:51.982686 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:51.982698 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:51.982835 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:51.982941 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:51.983460 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.984723 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.986802 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:51.987498 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:51.987635 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:51.988154 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:51.988332 (+ 178us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:51.988427 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:51.988640 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 18:11:51.988742 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
W20260807 18:11:52.026901 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.022314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.8.62.188:34553)
0807 18:11:52.022592 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.022598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.025130 (+ 2532us) server_negotiation.cc:407] Connection header received
0807 18:11:52.025328 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.025335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.025436 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.025578 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.026646 (+ 1068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57178: BlockingRecv error: recv got EOF from 127.0.0.1:57178 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
I20260807 18:11:52.047125 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.039707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59030 (local address 127.8.62.189:45895)
0807 18:11:52.039904 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.039910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.040441 (+ 531us) server_negotiation.cc:407] Connection header received
0807 18:11:52.040687 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.040693 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.040814 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.040920 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.041452 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.042687 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.044865 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.045555 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.045697 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.046222 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.046370 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.046460 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.046626 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.046728 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 18:11:52.056519 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.054564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.8.62.188:34553)
0807 18:11:52.054745 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.054751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.054773 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:11:52.054990 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.054997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.055106 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.055182 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.056283 (+ 1101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57186: BlockingRecv error: recv got EOF from 127.0.0.1:57186 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 18:11:52.083588 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.076313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59046 (local address 127.8.62.189:45895)
0807 18:11:52.076524 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.076532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.077187 (+ 655us) server_negotiation.cc:407] Connection header received
0807 18:11:52.077410 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.077418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.077518 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.077624 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.078120 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.079311 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.081361 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.082118 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.082240 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.082756 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.082880 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.082965 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.083183 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.083292 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260807 18:11:52.111882 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.108872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.8.62.188:34553)
0807 18:11:52.109069 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.109076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.110100 (+ 1024us) server_negotiation.cc:407] Connection header received
0807 18:11:52.110314 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.110321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.110432 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.110544 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.111660 (+ 1116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57190: BlockingRecv error: recv got EOF from 127.0.0.1:57190 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 18:11:52.138147 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7c2cdea99f534bcb8bcbde92db227162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c2cdea99f534bcb8bcbde92db227162" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 35605 } } peers { permanent_uuid: "a451ab5d90cc4835879833f5aafdd191" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 45895 } } peers { permanent_uuid: "86b1d510d95142b1b23bb581b33d9849" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 34553 } } }
I20260807 18:11:52.138525 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c2cdea99f534bcb8bcbde92db227162 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:52.157311 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.149178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.8.62.189:45895)
0807 18:11:52.149368 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.149374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.150740 (+ 1366us) server_negotiation.cc:407] Connection header received
0807 18:11:52.150929 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.150935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.151051 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.151158 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.151715 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.153005 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.154965 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.155632 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.155773 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.156380 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.156589 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.156702 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.156899 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.156993 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 18:11:52.169673 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.166744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57198 (local address 127.8.62.188:34553)
0807 18:11:52.166931 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.166937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.167588 (+ 651us) server_negotiation.cc:407] Connection header received
0807 18:11:52.168174 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.168181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.168286 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.168414 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.169391 (+ 977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57198: BlockingRecv error: recv got EOF from 127.0.0.1:57198 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 18:11:52.244256 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.237389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.8.62.189:45895)
0807 18:11:52.237646 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.237652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.238080 (+ 428us) server_negotiation.cc:407] Connection header received
0807 18:11:52.238339 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.238523 (+ 184us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.238624 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.238716 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.239256 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.240092 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.242137 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.242669 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.242777 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.243330 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.243457 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.243529 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.243737 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.243874 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260807 18:11:52.252604 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.250699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57214 (local address 127.8.62.188:34553)
0807 18:11:52.250904 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.250910 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.251204 (+ 294us) server_negotiation.cc:407] Connection header received
0807 18:11:52.251364 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.251370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.251481 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.251606 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.252341 (+ 735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57214: BlockingRecv error: recv got EOF from 127.0.0.1:57214 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 18:11:52.390705 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.384338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.8.62.189:45895)
0807 18:11:52.384657 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.384663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.385075 (+ 412us) server_negotiation.cc:407] Connection header received
0807 18:11:52.385305 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.385310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.385386 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.385488 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.386108 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.386807 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.388533 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.389010 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.389149 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.389849 (+ 700us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.389989 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.390077 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.390249 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.390322 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
W20260807 18:11:52.398914 31856 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:52.397058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.8.62.188:34553)
0807 18:11:52.397313 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.397319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.397580 (+ 261us) server_negotiation.cc:407] Connection header received
0807 18:11:52.397734 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.397738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.397810 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.397895 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:52.398644 (+ 749us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57224: BlockingRecv error: recv got EOF from 127.0.0.1:57224 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 18:11:52.643313 31852 catalog_manager.cc:1259] Loaded cluster ID: 55a5f18614224151987a5ffb036a8d4d
I20260807 18:11:52.643533 31852 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: loading cluster ID for follower catalog manager: success
I20260807 18:11:52.646498 31852 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: acquiring CA information for follower catalog manager: success
I20260807 18:11:52.648769 31852 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86b1d510d95142b1b23bb581b33d9849: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:11:52.666240 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.659302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.8.62.189:45895)
0807 18:11:52.659548 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.659555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.660019 (+ 464us) server_negotiation.cc:407] Connection header received
0807 18:11:52.660188 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.660196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.660300 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.660429 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.660975 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.662203 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.663612 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.664352 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.664501 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.665143 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.665318 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.665424 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.665854 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.665957 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 18:11:52.680313 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.673354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57230 (local address 127.8.62.188:34553)
0807 18:11:52.673543 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.673548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.673824 (+ 276us) server_negotiation.cc:407] Connection header received
0807 18:11:52.674022 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.674027 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.674183 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.674284 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.674812 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.675587 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.677793 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.678322 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.678418 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.679095 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.679640 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.679749 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.679913 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.679982 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:11:52.698473 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.687994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40240 (local address 127.8.62.190:35605)
0807 18:11:52.688596 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.688602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.689178 (+ 576us) server_negotiation.cc:407] Connection header received
0807 18:11:52.689363 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.689370 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.689598 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.689789 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.691168 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.692457 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.694932 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.695654 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.695790 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.696604 (+ 814us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.697114 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.697237 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.698009 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.698114 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":279,"threads_started":1}
I20260807 18:11:52.698995 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.689071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.8.62.189:45895)
0807 18:11:52.689313 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.689321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.689920 (+ 599us) server_negotiation.cc:407] Connection header received
0807 18:11:52.690260 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.690267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.690355 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.690443 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.691546 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.693040 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.695968 (+ 2928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.696908 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.697051 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.697915 (+ 864us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.698131 (+ 216us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.698208 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.698480 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.698589 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 18:11:52.705737 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:52.690236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.8.62.188:34553)
0807 18:11:52.690400 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:52.690407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:52.697903 (+ 7496us) server_negotiation.cc:407] Connection header received
0807 18:11:52.699106 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:52.699114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:52.699226 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:52.699298 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:52.700534 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.701860 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.703658 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:52.704226 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:52.704361 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:52.704918 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:52.705095 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:52.705220 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:52.705365 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 18:11:52.705450 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:11:52.705884 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31591
I20260807 18:11:52.707175 31653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:52.825299 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31591
I20260807 18:11:52.849054 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31665
I20260807 18:11:52.850363 31730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:52.961650 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31665
I20260807 18:11:52.996491 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 31743
I20260807 18:11:52.997810 31805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:53.121363 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 31743
2026-08-07T18:11:53Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4688 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 18:11:53 dist-test-slave-rnrw krb5kdc[32008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 18:11:53 dist-test-slave-rnrw krb5kdc[32008](info): set up 2 sockets
Aug 07 18:11:53 dist-test-slave-rnrw krb5kdc[32008](info): commencing operation
krb5kdc: starting...
W20260807 18:11:54.287948 8442 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/test-user.keytab.
Aug 07 18:11:54 dist-test-slave-rnrw krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T18:11:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T18:11:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.62.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
Entry for principal kudu/127.8.62.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab.
I20260807 18:11:54.588076 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:45165
--webserver_interface=127.8.62.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 18:11:54.937114 32021 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:54.937548 32021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:54.937690 32021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:54.937834 32021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:54.945284 32021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:54.945405 32021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:54.945541 32021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:54.945629 32021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:54.945739 32021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:54.955690 32021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.190.keytab
--principal=kudu/127.8.62.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.190:45165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.62.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:54.957154 32021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:54.958493 32021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:54.970852 32027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:54.970911 32029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:54.970916 32026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:54.971884 32021 server_base.cc:1034] running on GCE node
I20260807 18:11:54.972719 32021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:54.974213 32021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:54.975528 32021 hybrid_clock.cc:648] HybridClock initialized: now 1786126314975409 us; error 51 us; skew 500 ppm
I20260807 18:11:54.975893 32021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:54 dist-test-slave-rnrw krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:54.979246 32021 init.cc:377] Logged in from keytab as kudu/127.8.62.190@KRBTEST.COM (short username kudu)
I20260807 18:11:54.980975 32021 webserver.cc:460] Webserver started at http://127.8.62.190:45835/ using document root <none> and password file <none>
I20260807 18:11:54.981729 32021 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:54.981868 32021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:54.982201 32021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:54.986526 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data/instance:
uuid: "f1b79ba7088f4e929bab8b1c1181e797"
format_stamp: "Formatted at 2026-08-07 18:11:54 on dist-test-slave-rnrw"
server_key: "513f48d5e17b9820f8dd6ebcd43dbb5d"
server_key_iv: "d79aed344ac7b178b18abcb549efb7d2"
server_key_version: "encryptionkey@0"
I20260807 18:11:54.987373 32021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance:
uuid: "f1b79ba7088f4e929bab8b1c1181e797"
format_stamp: "Formatted at 2026-08-07 18:11:54 on dist-test-slave-rnrw"
server_key: "513f48d5e17b9820f8dd6ebcd43dbb5d"
server_key_iv: "d79aed344ac7b178b18abcb549efb7d2"
server_key_version: "encryptionkey@0"
I20260807 18:11:54.993072 32021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 18:11:54.996820 32036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:54.998270 32021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 18:11:54.998497 32021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal
uuid: "f1b79ba7088f4e929bab8b1c1181e797"
format_stamp: "Formatted at 2026-08-07 18:11:54 on dist-test-slave-rnrw"
server_key: "513f48d5e17b9820f8dd6ebcd43dbb5d"
server_key_iv: "d79aed344ac7b178b18abcb549efb7d2"
server_key_version: "encryptionkey@0"
I20260807 18:11:54.998795 32021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:55.014803 32021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:55.019536 32021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:55.019843 32021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:55.040225 32021 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.190:45165
I20260807 18:11:55.040248 32088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.190:45165 every 8 connection(s)
I20260807 18:11:55.041904 32021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/data/info.pb
I20260807 18:11:55.045778 32089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:55.046787 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 32021
I20260807 18:11:55.046998 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-0/wal/instance
I20260807 18:11:55.047600 8442 external_mini_cluster.cc:1428] Setting key 7b1562ffcb51b20ad2f74496fe179177
I20260807 18:11:55.056433 32089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:55 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.189@KRBTEST.COM" created.
I20260807 18:11:55.089568 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.061019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.8.62.190:45165)
0807 18:11:55.069135 (+ 8116us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:55.069142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:55.069179 (+ 37us) server_negotiation.cc:407] Connection header received
0807 18:11:55.069345 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:55.069350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:55.069666 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:55.069860 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:55.071769 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.073180 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.075593 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.076186 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.076341 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.080749 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:55.080761 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:55.080780 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:55.080823 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:55.084430 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.085351 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.085357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.085360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.085512 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.086162 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.086166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.086168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.086936 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:55.087152 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:55.087987 (+ 835us) server_negotiation.cc:299] Negotiation successful
0807 18:11:55.088291 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7902,"spinlock_wait_cycles":61184,"thread_start_us":7635,"threads_started":1}
I20260807 18:11:55.089615 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.061019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45165 (local address 127.0.0.1:38190)
0807 18:11:55.061991 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.062034 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:55.064016 (+ 1982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:55.069974 (+ 5958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:55.069988 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:55.070904 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:55.071487 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.071519 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.073366 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.073374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.075404 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.075422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.076447 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.076458 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.076723 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.077468 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:55.077505 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:55.080434 (+ 2929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:55.084712 (+ 4278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.084726 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.084742 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.085224 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.085786 (+ 562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.085790 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.085793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.086047 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.087109 (+ 1062us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:55.087137 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:55.087449 (+ 312us) client_negotiation.cc:769] Sending connection context
0807 18:11:55.087860 (+ 411us) client_negotiation.cc:240] Negotiation successful
0807 18:11:55.088593 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":75,"thread_start_us":442,"threads_started":1}
I20260807 18:11:55.093111 32089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:55.094404 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.093877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34620)
0807 18:11:55.094089 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.094238 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 18:11:55.094683 32038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
I20260807 18:11:55.096417 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.095950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34634)
0807 18:11:55.096132 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.096228 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:11:55.096649 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:42791: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
Entry for principal kudu/127.8.62.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab.
I20260807 18:11:55.124854 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:42791
--webserver_interface=127.8.62.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:11:55.146061 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } attempt: 1
I20260807 18:11:55.147671 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.147004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34646)
0807 18:11:55.147290 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.147438 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 18:11:55.149116 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.148582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34656)
0807 18:11:55.148806 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.148906 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 18:11:55.149430 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:42791: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
I20260807 18:11:55.217749 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } attempt: 2
I20260807 18:11:55.219210 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.218589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34666)
0807 18:11:55.218849 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.218995 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260807 18:11:55.220510 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.220054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34678)
0807 18:11:55.220226 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.220334 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 18:11:55.220825 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:42791: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
I20260807 18:11:55.312136 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } attempt: 3
I20260807 18:11:55.313573 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.313028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34688)
0807 18:11:55.313317 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.313412 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 18:11:55.314703 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.314316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34704)
0807 18:11:55.314470 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.314560 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 18:11:55.314888 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:42791: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
I20260807 18:11:55.458156 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } attempt: 4
I20260807 18:11:55.459375 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.458829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34714)
0807 18:11:55.459153 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.459242 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260807 18:11:55.460469 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.460117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34728)
0807 18:11:55.460276 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.460343 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 18:11:55.460659 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.189:42791: Network error: Client connection negotiation failed: client connection to 127.8.62.189:42791: connect: Connection refused (error 111)
W20260807 18:11:55.483753 32096 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:55.484150 32096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:55.484280 32096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:55.484431 32096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:55.491816 32096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:55.491940 32096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:55.492053 32096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:55.492153 32096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:55.492246 32096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:55.502910 32096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.189.keytab
--principal=kudu/127.8.62.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.189:42791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.62.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:55.504339 32096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:55.505681 32096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:55.517752 32101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:55.517840 32104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:55.517874 32102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:55.519215 32096 server_base.cc:1034] running on GCE node
I20260807 18:11:55.520015 32096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:55.521575 32096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:55.522842 32096 hybrid_clock.cc:648] HybridClock initialized: now 1786126315522775 us; error 47 us; skew 500 ppm
I20260807 18:11:55.523212 32096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:55 dist-test-slave-rnrw krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126315, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:55.526597 32096 init.cc:377] Logged in from keytab as kudu/127.8.62.189@KRBTEST.COM (short username kudu)
I20260807 18:11:55.528246 32096 webserver.cc:460] Webserver started at http://127.8.62.189:34845/ using document root <none> and password file <none>
I20260807 18:11:55.529237 32096 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:55.529386 32096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:55.529731 32096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:55.532852 32096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data/instance:
uuid: "ed066b097ae74ae4934ce422af8c49ef"
format_stamp: "Formatted at 2026-08-07 18:11:55 on dist-test-slave-rnrw"
server_key: "019bd6b8fb324904bc6769a8bb77eec3"
server_key_iv: "936e3528414a7ceb79c4c271cbc8b89c"
server_key_version: "encryptionkey@0"
I20260807 18:11:55.533739 32096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal/instance:
uuid: "ed066b097ae74ae4934ce422af8c49ef"
format_stamp: "Formatted at 2026-08-07 18:11:55 on dist-test-slave-rnrw"
server_key: "019bd6b8fb324904bc6769a8bb77eec3"
server_key_iv: "936e3528414a7ceb79c4c271cbc8b89c"
server_key_version: "encryptionkey@0"
I20260807 18:11:55.539305 32096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 18:11:55.542937 32111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:55.544241 32096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260807 18:11:55.544435 32096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal
uuid: "ed066b097ae74ae4934ce422af8c49ef"
format_stamp: "Formatted at 2026-08-07 18:11:55 on dist-test-slave-rnrw"
server_key: "019bd6b8fb324904bc6769a8bb77eec3"
server_key_iv: "936e3528414a7ceb79c4c271cbc8b89c"
server_key_version: "encryptionkey@0"
I20260807 18:11:55.544735 32096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:55.579355 32096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:55.583804 32096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:55.584105 32096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:55.603855 32096 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.189:42791
I20260807 18:11:55.603870 32164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.189:42791 every 8 connection(s)
I20260807 18:11:55.605357 32096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/data/info.pb
I20260807 18:11:55.609087 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 32096
I20260807 18:11:55.609303 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-1/wal/instance
I20260807 18:11:55.609486 32165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:55.609828 8442 external_mini_cluster.cc:1428] Setting key 2bb1fc92d118632e964d4382915dc4e9
I20260807 18:11:55.616755 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:55 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126315, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.62.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.62.188@KRBTEST.COM" created.
I20260807 18:11:55.640800 32168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.619770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.8.62.190:45165)
0807 18:11:55.620408 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:55.620417 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:55.621463 (+ 1046us) server_negotiation.cc:407] Connection header received
0807 18:11:55.623446 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:55.623454 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:55.623566 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:55.623717 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:55.625405 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.626711 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.628404 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.629227 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.629376 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.633372 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:55.633381 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:55.633389 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:55.633454 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:55.637417 (+ 3963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.638337 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.638341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.638343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.638466 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.638933 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.638936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.638938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.639231 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:55.639347 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:55.640161 (+ 814us) server_negotiation.cc:299] Negotiation successful
0807 18:11:55.640336 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":311,"threads_started":1}
I20260807 18:11:55.641865 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.620347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45165 (local address 127.0.0.1:38196)
0807 18:11:55.621257 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.621318 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:55.623153 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:55.623918 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:55.623926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:55.624502 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:55.625171 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.625189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.626908 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.626912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.628237 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.628249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.629461 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.629468 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.629678 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.630434 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:55.630460 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:55.633106 (+ 2646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:55.637626 (+ 4520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.637633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.637648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.638195 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.638629 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.638633 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.638635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.638811 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.639354 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:55.639360 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:55.639759 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 18:11:55.640064 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 18:11:55.640625 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":426,"threads_started":1}
I20260807 18:11:55.643707 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:55 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126315, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:55.661777 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.644615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34734)
0807 18:11:55.644840 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.644861 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:55.644971 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:55.646097 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:55.646100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:55.646464 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:55.646668 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.646677 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.648794 (+ 2117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.648802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.650340 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.650358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.651302 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.651311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.651550 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.652223 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:55.652271 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:55.654505 (+ 2234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:55.659231 (+ 4726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.659236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.659238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.659603 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.660042 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.660045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.660047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.660201 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.661018 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:55.661021 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:55.661176 (+ 155us) client_negotiation.cc:769] Sending connection context
0807 18:11:55.661296 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 18:11:55.661512 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:11:55.662300 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.644862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.8.62.189:42791)
0807 18:11:55.645570 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:55.645580 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:55.645611 (+ 31us) server_negotiation.cc:407] Connection header received
0807 18:11:55.645670 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:55.645680 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:55.645924 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:55.646089 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:55.646849 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.648641 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.650548 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.651129 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.651236 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.654735 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:55.654754 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:55.654779 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:55.654823 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:55.659064 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.659749 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.659761 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.659767 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.659927 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.660311 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.660319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.660324 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.660874 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:55.661144 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:55.661496 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 18:11:55.661801 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":358,"threads_started":1}
I20260807 18:11:55.665096 32165 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:55.666404 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.665907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42904)
0807 18:11:55.666110 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.666243 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 18:11:55.666714 32115 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.668416 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.668070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42914)
0807 18:11:55.668235 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.668314 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:11:55.668685 32165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
Entry for principal kudu/127.8.62.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab.
I20260807 18:11:55.684005 8442 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
/tmp/dist-test-taskn0cCOk/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:44319
--webserver_interface=127.8.62.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.62.148:35971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 18:11:55.717985 32165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 1
I20260807 18:11:55.719302 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.718753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42928)
0807 18:11:55.719016 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.719148 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 18:11:55.720325 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.719978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42940)
0807 18:11:55.720125 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.720221 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 18:11:55.720597 32165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.759923 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } attempt: 5
Aug 07 18:11:55 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:55.775215 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.760824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.8.62.189:42791)
0807 18:11:55.761018 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:55.761024 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:55.761049 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:11:55.761207 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:55.761213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:55.761316 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:55.761463 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:55.762293 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.763369 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.764597 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.765275 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.765407 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.768728 (+ 3321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:55.768737 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:55.768744 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:55.768795 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:55.772328 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.773262 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.773267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.773269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.773405 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:55.773832 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:55.773836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:55.773838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:55.774140 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:55.774252 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:55.774757 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 18:11:55.774988 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:11:55.775278 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.760734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34736)
0807 18:11:55.760932 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.760956 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:55.761072 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:55.761488 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:55.761491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:55.761932 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:55.762121 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.762131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.763528 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.763532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.764412 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:55.764423 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:55.765429 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:55.765433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:55.765627 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:55.766359 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:55.766378 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:55.768474 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:55.772529 (+ 4055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.772535 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.772572 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.773123 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.773531 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:55.773537 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:55.773540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:55.773712 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:55.774265 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:55.774271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:55.774454 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 18:11:55.774637 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 18:11:55.774941 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 18:11:55.776495 32089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:55.777820 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.777399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42950)
0807 18:11:55.777596 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.777674 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 18:11:55.778864 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.778571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42964)
0807 18:11:55.778694 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.778759 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:11:55.779156 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.788816 32165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 2
I20260807 18:11:55.789870 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.789433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42972)
0807 18:11:55.789626 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.789718 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 18:11:55.790834 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.790482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42982)
0807 18:11:55.790667 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.790732 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 18:11:55.791069 32165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.830400 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 1
I20260807 18:11:55.831686 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.831183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42988)
0807 18:11:55.831410 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.831524 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:11:55.832875 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.832488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42996)
0807 18:11:55.832688 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.832755 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260807 18:11:55.833148 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.882323 32165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 3
I20260807 18:11:55.883520 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.883027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:42998)
0807 18:11:55.883275 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.883362 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 18:11:55.884707 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.884197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43010)
0807 18:11:55.884423 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.884499 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260807 18:11:55.884922 32165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:55.901496 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 2
I20260807 18:11:55.902842 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.902256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43012)
0807 18:11:55.902497 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.902615 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 18:11:55.903929 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:55.903548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43018)
0807 18:11:55.903710 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:55.903782 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 18:11:55.904181 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:56.010389 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 3
I20260807 18:11:56.011484 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.010997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43030)
0807 18:11:56.011221 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.011316 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 18:11:56.012501 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.012135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43032)
0807 18:11:56.012340 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.012403 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 18:11:56.012771 32089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
I20260807 18:11:56.028160 32165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 4
I20260807 18:11:56.029263 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.028803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43046)
0807 18:11:56.029030 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.029118 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 18:11:56.030201 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.029887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43058)
0807 18:11:56.030041 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.030104 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 18:11:56.030442 32165 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.62.188:44319: Network error: Client connection negotiation failed: client connection to 127.8.62.188:44319: connect: Connection refused (error 111)
W20260807 18:11:56.033800 32173 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 18:11:56.034184 32173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 18:11:56.034315 32173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 18:11:56.034466 32173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 18:11:56.042021 32173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 18:11:56.042135 32173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 18:11:56.042232 32173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 18:11:56.042299 32173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 18:11:56.042388 32173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 18:11:56.052356 32173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.62.148:35971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.62.190:45165,127.8.62.189:42791,127.8.62.188:44319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/krb5kdc/kudu_127.8.62.188.keytab
--principal=kudu/127.8.62.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.62.188:44319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.62.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 16:43:01 UTC on ccycloud.quasar-himzip.root.comops.site
build id 4572595
ASAN enabled
I20260807 18:11:56.053639 32173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 18:11:56.054909 32173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 18:11:56.066865 32178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:56.066839 32179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 18:11:56.066974 32181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 18:11:56.067875 32173 server_base.cc:1034] running on GCE node
I20260807 18:11:56.068742 32173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 18:11:56.070191 32173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 18:11:56.071494 32173 hybrid_clock.cc:648] HybridClock initialized: now 1786126316071409 us; error 77 us; skew 500 ppm
I20260807 18:11:56.071863 32173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 18:11:56.075093 32173 init.cc:377] Logged in from keytab as kudu/127.8.62.188@KRBTEST.COM (short username kudu)
I20260807 18:11:56.076529 32173 webserver.cc:460] Webserver started at http://127.8.62.188:38231/ using document root <none> and password file <none>
I20260807 18:11:56.077276 32173 fs_manager.cc:359] Metadata directory not provided
I20260807 18:11:56.077406 32173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 18:11:56.077720 32173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 18:11:56.080729 32173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data/instance:
uuid: "16c0f799be4842b6b3aa7fef022f6347"
format_stamp: "Formatted at 2026-08-07 18:11:56 on dist-test-slave-rnrw"
server_key: "534ade2c9257ea50e11c3d0c92c6f62a"
server_key_iv: "8f8a3cf72c8d502e6afc00e69e644f82"
server_key_version: "encryptionkey@0"
I20260807 18:11:56.081508 32173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal/instance:
uuid: "16c0f799be4842b6b3aa7fef022f6347"
format_stamp: "Formatted at 2026-08-07 18:11:56 on dist-test-slave-rnrw"
server_key: "534ade2c9257ea50e11c3d0c92c6f62a"
server_key_iv: "8f8a3cf72c8d502e6afc00e69e644f82"
server_key_version: "encryptionkey@0"
I20260807 18:11:56.086742 32173 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 18:11:56.090229 32188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 18:11:56.091710 32173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 18:11:56.091898 32173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data,/tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal
uuid: "16c0f799be4842b6b3aa7fef022f6347"
format_stamp: "Formatted at 2026-08-07 18:11:56 on dist-test-slave-rnrw"
server_key: "534ade2c9257ea50e11c3d0c92c6f62a"
server_key_iv: "8f8a3cf72c8d502e6afc00e69e644f82"
server_key_version: "encryptionkey@0"
I20260807 18:11:56.092129 32173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 18:11:56.111037 32173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 18:11:56.116045 32173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 18:11:56.116372 32173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 18:11:56.135140 32173 rpc_server.cc:307] RPC server started. Bound to: 127.8.62.188:44319
I20260807 18:11:56.135161 32240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.62.188:44319 every 8 connection(s)
I20260807 18:11:56.136693 32173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/data/info.pb
I20260807 18:11:56.140800 32241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 18:11:56.143071 8442 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu as pid 32173
I20260807 18:11:56.143286 8442 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskn0cCOk/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786125932226923-8442-0/minicluster-data/master-2/wal/instance
I20260807 18:11:56.143728 8442 external_mini_cluster.cc:1428] Setting key 7960f406b87dc07acb361726b8ecdc00
I20260807 18:11:56.150224 32241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:56.169659 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.145719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.8.62.188:44319)
0807 18:11:56.146609 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.146626 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.146688 (+ 62us) server_negotiation.cc:407] Connection header received
0807 18:11:56.149225 (+ 2537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.149260 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.149853 (+ 593us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.150358 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.152175 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.153938 (+ 1763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.155582 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.156323 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.156531 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.161888 (+ 5357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.161904 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.161921 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.161958 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.165230 (+ 3272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.165988 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.166004 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.166011 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.166180 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.166584 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.166590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.166593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.167164 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.167504 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.168016 (+ 512us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.168520 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":361,"threads_started":1}
I20260807 18:11:56.171207 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.153385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.8.62.190:45165)
0807 18:11:56.153954 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.153960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.154620 (+ 660us) server_negotiation.cc:407] Connection header received
0807 18:11:56.154847 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.154853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.154937 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.155050 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.156383 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.157273 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.159548 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.160250 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.160404 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.164691 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.164697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.164700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.164739 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.167874 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.168893 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.168898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.168904 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.169056 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.169489 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.169499 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.169501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.169762 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.169934 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.170519 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.170766 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":277,"threads_started":1}
I20260807 18:11:56.171234 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.153385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.190:45165 (local address 127.0.0.1:38212)
0807 18:11:56.154448 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.154501 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:56.154691 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:56.155158 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:56.155164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:56.155865 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:56.156183 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.156193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.157462 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.157468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.159275 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.159293 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.160496 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.160501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.160740 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.161437 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:56.161461 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:56.164393 (+ 2932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:56.168026 (+ 3633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.168042 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.168050 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.168748 (+ 698us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.169181 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.169186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.169190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.169353 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.170019 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:56.170027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:56.170202 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 18:11:56.170401 (+ 199us) client_negotiation.cc:240] Negotiation successful
0807 18:11:56.170770 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":577,"threads_started":1}
I20260807 18:11:56.172816 32241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.190@KRBTEST.COM
I20260807 18:11:56.188222 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.173533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.189:42791 (local address 127.0.0.1:34748)
0807 18:11:56.173696 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.173714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:56.173810 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:56.174215 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:56.174218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:56.174623 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:56.174780 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.174788 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.175942 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.175947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.177069 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.177085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.178162 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.178168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.178365 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.179163 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:56.179182 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:56.181463 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:56.185518 (+ 4055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.185522 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.185525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.185920 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.186431 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.186435 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.186438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.186688 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.187277 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:56.187284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:56.187471 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 18:11:56.187643 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 18:11:56.187965 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 18:11:56.188354 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.173650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.8.62.189:42791)
0807 18:11:56.173847 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.173852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.173871 (+ 19us) server_negotiation.cc:407] Connection header received
0807 18:11:56.173942 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.173947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.174033 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.174147 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.174982 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.175785 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.177234 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.177986 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.178110 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.181709 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.181714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.181717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.181754 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.185295 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.186061 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.186067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.186072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.186256 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.186821 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.186825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.186827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.187132 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.187253 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.187746 (+ 493us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.187976 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:11:56.189324 32241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } has no permanent_uuid. Determining permanent_uuid...
I20260807 18:11:56.190001 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.173091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.8.62.190:45165)
0807 18:11:56.173261 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.173267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.173284 (+ 17us) server_negotiation.cc:407] Connection header received
0807 18:11:56.173479 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.173484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.173577 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.173669 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.175267 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.176082 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.177898 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.178456 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.178601 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.183252 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.183261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.183267 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.183314 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.186573 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.187363 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.187367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.187369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.187508 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.187923 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.187928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.187932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.188295 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.188871 (+ 576us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.189200 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.189444 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 18:11:56.189972 32089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 4
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126316, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.188@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.190@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126314, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.62.189@KRBTEST.COM
I20260807 18:11:56.207669 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.190230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43062)
0807 18:11:56.190597 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.190612 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:56.190693 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:56.191878 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:56.191886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:56.192586 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:56.192906 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.192921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.194208 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.194215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.195259 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.195275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.196718 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.196722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.196865 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.197519 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:56.197552 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:56.200366 (+ 2814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:56.204342 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.204347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.204351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.204867 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.205299 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.205304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.205307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.205470 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.206326 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:56.206334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:56.206582 (+ 248us) client_negotiation.cc:769] Sending connection context
0807 18:11:56.206746 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 18:11:56.207205 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246}
I20260807 18:11:56.208156 32244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.191222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.8.62.188:44319)
0807 18:11:56.191397 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.191404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.191424 (+ 20us) server_negotiation.cc:407] Connection header received
0807 18:11:56.191484 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.191491 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.191569 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.191675 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.193064 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.194053 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.195411 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.196001 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.196135 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.200643 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.200649 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.200654 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.200695 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.204175 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.204996 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.205002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.205005 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.205157 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.205606 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.205615 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.205621 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.206190 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.206349 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.206947 (+ 598us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.207418 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":66}
I20260807 18:11:56.209383 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.190541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43066)
0807 18:11:56.190791 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.190812 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:56.190915 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:56.193550 (+ 2635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:56.193555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:56.194036 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:56.194260 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.194274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.195845 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.195850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.197415 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.197431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.198434 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.198439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.198682 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.199448 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:56.199476 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:56.202100 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:56.206403 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.206408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.206412 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.206937 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.207549 (+ 612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.207554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.207557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.207794 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.208469 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:56.208474 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:56.208699 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 18:11:56.208826 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 18:11:56.209088 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260807 18:11:56.210394 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.191358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.8.62.188:44319)
0807 18:11:56.193189 (+ 1831us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.193196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.193219 (+ 23us) server_negotiation.cc:407] Connection header received
0807 18:11:56.193276 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.193282 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.193361 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.193490 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.194439 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.195704 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.197568 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.198227 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.199029 (+ 802us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.202414 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.202422 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.202427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.202472 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.206226 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.207208 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.207212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.207214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.207384 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.207941 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.207946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.207949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.208306 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.209659 (+ 1353us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.209774 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.209977 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1673,"thread_start_us":1454,"threads_started":1}
I20260807 18:11:56.211076 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.191840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.8.62.189:42791)
0807 18:11:56.191996 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.192001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.192101 (+ 100us) server_negotiation.cc:407] Connection header received
0807 18:11:56.192247 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.192252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.192320 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.192423 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.194179 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.195295 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.197931 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.198366 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.198523 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.203871 (+ 5348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.203881 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.203886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.203926 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.207153 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.208157 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.208164 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.208167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.208333 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.208865 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.208870 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.208872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.209257 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.209574 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.210142 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.210391 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 18:11:56.214804 32241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: Bootstrap starting.
I20260807 18:11:56.215945 8442 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 18:11:56.216032 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: Bootstrap starting.
I20260807 18:11:56.219213 32241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:56.220192 32089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:56.220858 32241 log.cc:824] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:56.218357 32064 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f1b79ba7088f4e929bab8b1c1181e797"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38216
I20260807 18:11:56.221732 32089 log.cc:824] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:56.224308 32089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: No bootstrap required, opened a new log
I20260807 18:11:56.224359 32241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: No bootstrap required, opened a new log
I20260807 18:11:56.229838 32089 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.230203 32089 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b79ba7088f4e929bab8b1c1181e797, State: Initialized, Role: FOLLOWER
I20260807 18:11:56.230674 32089 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.231704 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.232141 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.232980 32089 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:56.233103 32064 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 18:11:56.233311 32064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:56.235262 32241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.235703 32241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16c0f799be4842b6b3aa7fef022f6347, State: Initialized, Role: FOLLOWER
I20260807 18:11:56.236258 32241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.237012 32064 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.237485 32256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.238147 32256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.238726 32241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:56.239593 32216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1b79ba7088f4e929bab8b1c1181e797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "16c0f799be4842b6b3aa7fef022f6347"
I20260807 18:11:56.240114 32216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:56.244967 32216 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1b79ba7088f4e929bab8b1c1181e797 in term 1.
I20260807 18:11:56.245723 32064 leader_election.cc:288] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [CANDIDATE]: Term 1 election: Requested vote from peers ed066b097ae74ae4934ce422af8c49ef (127.8.62.189:42791), 16c0f799be4842b6b3aa7fef022f6347 (127.8.62.188:44319)
I20260807 18:11:56.245927 32040 leader_election.cc:302] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16c0f799be4842b6b3aa7fef022f6347, f1b79ba7088f4e929bab8b1c1181e797; no voters:
I20260807 18:11:56.246567 32250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 18:11:56.259431 32250 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [term 1 LEADER]: Becoming Leader. State: Replica: f1b79ba7088f4e929bab8b1c1181e797, State: Running, Role: LEADER
I20260807 18:11:56.260214 32250 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
W20260807 18:11:56.272965 32042 tablet.cc:2369] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:56.276377 32264 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b79ba7088f4e929bab8b1c1181e797. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.276952 32264 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:56.277034 32276 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 18:11:56.278851 32277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 18:11:56.297550 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.258416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.8.62.190:45165)
0807 18:11:56.258674 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.258681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.264217 (+ 5536us) server_negotiation.cc:407] Connection header received
0807 18:11:56.264875 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.264884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.265001 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.265086 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.269802 (+ 4716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.271190 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.276494 (+ 5304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.277353 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.277575 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.284713 (+ 7138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.284722 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.284730 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.284784 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.288658 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.289569 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.289576 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.289579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.289782 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.294931 (+ 5149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.294938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.294941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.295705 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.296085 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.296787 (+ 702us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.297053 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 18:11:56.299418 32277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 18:11:56.282338 32021 master_runner.cc:186] Error getting master registration for 127.8.62.189:42791: OK, instance_id { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" instance_seqno: 1786126315598125 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:11:56.301136 32021 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:11:56.306380 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.264037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.8.62.188:44319)
0807 18:11:56.264263 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.264272 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.270390 (+ 6118us) server_negotiation.cc:407] Connection header received
0807 18:11:56.270945 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.270956 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.271123 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.271216 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.275532 (+ 4316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.276903 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.281637 (+ 4734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.282329 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.282485 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.293457 (+ 10972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.293473 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.293479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.293536 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.297832 (+ 4296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.303078 (+ 5246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.303085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.303089 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.303342 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.303869 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.303879 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.303882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.304297 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.304618 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.305312 (+ 694us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.305763 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:11:56.308346 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.261862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.8.62.189:42791)
0807 18:11:56.262108 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.262117 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.265501 (+ 3384us) server_negotiation.cc:407] Connection header received
0807 18:11:56.266517 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.266525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.266619 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.266703 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.270812 (+ 4109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.271863 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.289624 (+ 17761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.290384 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.290506 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.299677 (+ 9171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.299685 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.299689 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.299741 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.303335 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.305381 (+ 2046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.305387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.305390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.305593 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.306071 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.306078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.306083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.306496 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.306736 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.307677 (+ 941us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.307941 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 18:11:56.314455 32216 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [term 1 FOLLOWER]: Refusing update from remote peer f1b79ba7088f4e929bab8b1c1181e797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:56.315455 32250 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:56.332093 32165 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } attempt: 5
W20260807 18:11:56.332986 32287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 18:11:56.333621 32287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 07 18:11:56 dist-test-slave-rnrw krb5kdc[32008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786126315, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.62.189@KRBTEST.COM for kudu/127.8.62.188@KRBTEST.COM
I20260807 18:11:56.359675 32169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.332746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.62.188:44319 (local address 127.0.0.1:43088)
0807 18:11:56.333145 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 18:11:56.333167 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 18:11:56.333285 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 18:11:56.334237 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 18:11:56.334243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 18:11:56.334804 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 18:11:56.335042 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.335054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.336728 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.336734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.338164 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 18:11:56.338179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.339534 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.339550 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 18:11:56.339758 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.340577 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 18:11:56.340604 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 18:11:56.346978 (+ 6374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 18:11:56.353740 (+ 6762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.353746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.353751 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.354394 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.357127 (+ 2733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 18:11:56.357134 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 18:11:56.357137 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 18:11:56.357372 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 18:11:56.358289 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 18:11:56.358294 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 18:11:56.358505 (+ 211us) client_negotiation.cc:769] Sending connection context
0807 18:11:56.358700 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 18:11:56.359209 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255}
I20260807 18:11:56.361806 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.333494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.8.62.188:44319)
0807 18:11:56.333683 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.333689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.333714 (+ 25us) server_negotiation.cc:407] Connection header received
0807 18:11:56.333774 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.333780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.333869 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.333980 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.335221 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.336487 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.338349 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.339089 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.339219 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.348782 (+ 9563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 18:11:56.348792 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 18:11:56.348800 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 18:11:56.348860 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 18:11:56.353448 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.356660 (+ 3212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.356668 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.356672 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.356905 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 18:11:56.357555 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 18:11:56.357561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 18:11:56.357565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 18:11:56.358004 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 18:11:56.358200 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.360174 (+ 1974us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.360471 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 18:11:56.368578 32194 tablet.cc:2369] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 18:11:56.368901 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: Bootstrap starting.
I20260807 18:11:56.374501 32165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: Neither blocks nor log segments found. Creating new log.
I20260807 18:11:56.327718 32173 master_runner.cc:186] Error getting master registration for 127.8.62.189:42791: OK, instance_id { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" instance_seqno: 1786126315598125 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 18:11:56.380901 32165 log.cc:824] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: Log is configured to *not* fsync() on all Append() calls
I20260807 18:11:56.381345 32173 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 18:11:56.384375 32256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b79ba7088f4e929bab8b1c1181e797. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.384788 32256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.388305 32283 mvcc.cc:205] Tried to move back new op lower bound from 7315973391609647104 to 7315973391408308224. Current Snapshot: MvccSnapshot[applied={T|T < 7315973391609647104}]
I20260807 18:11:56.390894 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.391249 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:56.394090 32277 catalog_manager.cc:1347] Generated new cluster ID: 6dd85dd92ac143d2a14f47bef77d5c55
I20260807 18:11:56.394215 32277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 18:11:56.399487 32165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: No bootstrap required, opened a new log
I20260807 18:11:56.403100 32256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.403391 32256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.413789 32165 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.414268 32165 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed066b097ae74ae4934ce422af8c49ef, State: Initialized, Role: FOLLOWER
I20260807 18:11:56.414810 32165 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } }
I20260807 18:11:56.416239 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.417022 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.417259 32165 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: configured and running, proceeding with master startup.
I20260807 18:11:56.417694 32137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [term 0 FOLLOWER]: Advancing to term 1
I20260807 18:11:56.421871 32137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [term 1 FOLLOWER]: Refusing update from remote peer f1b79ba7088f4e929bab8b1c1181e797: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 18:11:56.422544 32138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f1b79ba7088f4e929bab8b1c1181e797" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ed066b097ae74ae4934ce422af8c49ef"
I20260807 18:11:56.426401 32138 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1b79ba7088f4e929bab8b1c1181e797 in term 1.
I20260807 18:11:56.426668 32264 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 18:11:56.434378 32277 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 18:11:56.436412 32277 catalog_manager.cc:1504] Loading token signing keys...
I20260807 18:11:56.480463 32299 mvcc.cc:205] Tried to move back new op lower bound from 7315973391609647104 to 7315973391408308224. Current Snapshot: MvccSnapshot[applied={T|T < 7315973391609647104}]
I20260807 18:11:56.495483 32298 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.495864 32250 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.496064 32298 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.496156 32250 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797 [sys.catalog]: This master's current role is: LEADER
I20260807 18:11:56.496567 32296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b79ba7088f4e929bab8b1c1181e797. Latest consensus state: current_term: 1 leader_uuid: "f1b79ba7088f4e929bab8b1c1181e797" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1b79ba7088f4e929bab8b1c1181e797" member_type: VOTER last_known_addr { host: "127.8.62.190" port: 45165 } } peers { permanent_uuid: "ed066b097ae74ae4934ce422af8c49ef" member_type: VOTER last_known_addr { host: "127.8.62.189" port: 42791 } } peers { permanent_uuid: "16c0f799be4842b6b3aa7fef022f6347" member_type: VOTER last_known_addr { host: "127.8.62.188" port: 44319 } } }
I20260807 18:11:56.496886 32296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef [sys.catalog]: This master's current role is: FOLLOWER
I20260807 18:11:56.527019 32277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1b79ba7088f4e929bab8b1c1181e797: Generated new TSK 0
I20260807 18:11:56.528308 32277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 18:11:56.580394 32311 catalog_manager.cc:1259] Loaded cluster ID: 6dd85dd92ac143d2a14f47bef77d5c55
I20260807 18:11:56.580646 32311 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: loading cluster ID for follower catalog manager: success
I20260807 18:11:56.585357 32311 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: acquiring CA information for follower catalog manager: success
I20260807 18:11:56.588429 32311 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed066b097ae74ae4934ce422af8c49ef: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 18:11:56.696192 32245 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.668702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.8.62.190:45165)
0807 18:11:56.668949 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.668956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.692184 (+ 23228us) server_negotiation.cc:407] Connection header received
0807 18:11:56.693982 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.693989 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.694077 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.694152 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.695910 (+ 1758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38228: BlockingRecv error: recv got EOF from 127.0.0.1:38228 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:11:56.697459 32170 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.675473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.8.62.189:42791)
0807 18:11:56.675702 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.675708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.692708 (+ 17000us) server_negotiation.cc:407] Connection header received
0807 18:11:56.693483 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.693490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.693626 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.693714 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.697226 (+ 3512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34764: BlockingRecv error: recv got EOF from 127.0.0.1:34764 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 18:11:56.700587 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.689526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43100 (local address 127.8.62.188:44319)
0807 18:11:56.689751 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.689758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.698239 (+ 8481us) server_negotiation.cc:407] Connection header received
0807 18:11:56.699143 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.699149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.699241 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.699312 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.700303 (+ 991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43100: BlockingRecv error: recv got EOF from 127.0.0.1:43100 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 18:11:56.750129 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.730402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.8.62.189:42791)
0807 18:11:56.730597 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.730605 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.741258 (+ 10653us) server_negotiation.cc:407] Connection header received
0807 18:11:56.741846 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.741853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.742116 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.742272 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:56.743099 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.744304 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.746419 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.747127 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.747239 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.748247 (+ 1008us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:56.749118 (+ 871us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:56.749255 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.749594 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.749723 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 18:11:56.795471 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.785504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.8.62.188:44319)
0807 18:11:56.785734 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.785741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.793246 (+ 7505us) server_negotiation.cc:407] Connection header received
0807 18:11:56.793822 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.793830 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.793990 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.794098 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.795220 (+ 1122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43110: BlockingRecv error: recv got EOF from 127.0.0.1:43110 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 18:11:56.849584 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.831077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34784 (local address 127.8.62.189:42791)
0807 18:11:56.831293 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.831300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.842242 (+ 10942us) server_negotiation.cc:407] Connection header received
0807 18:11:56.842599 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.842607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.842741 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.842814 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:56.843645 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.844871 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.847010 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.847680 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.847787 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.848525 (+ 738us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:56.848798 (+ 273us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:56.848921 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.849131 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.849230 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 18:11:56.893849 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.884292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43124 (local address 127.8.62.188:44319)
0807 18:11:56.884497 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.884508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.891508 (+ 7000us) server_negotiation.cc:407] Connection header received
0807 18:11:56.892112 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.892121 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.892235 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.892329 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.893574 (+ 1245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43124: BlockingRecv error: recv got EOF from 127.0.0.1:43124 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 18:11:56.952208 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:56.933648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.8.62.189:42791)
0807 18:11:56.933860 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.933869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.944572 (+ 10703us) server_negotiation.cc:407] Connection header received
0807 18:11:56.945142 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.945150 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.945254 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.945325 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:56.946119 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.947247 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.949724 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:56.950430 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:56.950541 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:56.951214 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:56.951424 (+ 210us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:56.951500 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:56.951748 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 18:11:56.951843 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:11:56.995271 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:56.985798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.8.62.188:44319)
0807 18:11:56.986026 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:56.986033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:56.993002 (+ 6969us) server_negotiation.cc:407] Connection header received
0807 18:11:56.993563 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:56.993572 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:56.993695 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:56.993774 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:56.995011 (+ 1237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43132: BlockingRecv error: recv got EOF from 127.0.0.1:43132 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:11:57.029490 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.022177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.8.62.189:42791)
0807 18:11:57.022383 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.022390 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.023217 (+ 827us) server_negotiation.cc:407] Connection header received
0807 18:11:57.023437 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.023443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.023543 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.023661 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.024150 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.025459 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.027380 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.028073 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.028220 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.028698 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.028836 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.028942 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.029107 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.029192 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 18:11:57.041502 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:57.036395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.8.62.188:44319)
0807 18:11:57.036699 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.036707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.039559 (+ 2852us) server_negotiation.cc:407] Connection header received
0807 18:11:57.039884 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.039892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.040043 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.040134 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:57.041274 (+ 1140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43134: BlockingRecv error: recv got EOF from 127.0.0.1:43134 (error 108)
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 18:11:57.079819 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.069730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.8.62.189:42791)
0807 18:11:57.069968 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.069975 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.073255 (+ 3280us) server_negotiation.cc:407] Connection header received
0807 18:11:57.073486 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.073493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.073596 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.073713 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.074439 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.075629 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.077590 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.078277 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.078428 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.078956 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.079127 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.079227 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.079397 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.079487 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 18:11:57.090086 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:57.087782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.8.62.188:44319)
0807 18:11:57.088009 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.088017 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.088315 (+ 298us) server_negotiation.cc:407] Connection header received
0807 18:11:57.088509 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.088515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.088639 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.088754 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:57.089844 (+ 1090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43144: BlockingRecv error: recv got EOF from 127.0.0.1:43144 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 18:11:57.118227 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.110503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.8.62.189:42791)
0807 18:11:57.110723 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.110730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.111101 (+ 371us) server_negotiation.cc:407] Connection header received
0807 18:11:57.111275 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.111282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.111384 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.111514 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.111952 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.113169 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.115635 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.116457 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.116653 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.117298 (+ 645us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.117443 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.117541 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.117827 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.117924 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260807 18:11:57.132819 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:57.130523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.8.62.188:44319)
0807 18:11:57.130743 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.130749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.131065 (+ 316us) server_negotiation.cc:407] Connection header received
0807 18:11:57.131253 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.131258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.131337 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.131448 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:57.132456 (+ 1008us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43156: BlockingRecv error: recv got EOF from 127.0.0.1:43156 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 18:11:57.175004 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.168819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34846 (local address 127.8.62.189:42791)
0807 18:11:57.169130 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.169139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.169401 (+ 262us) server_negotiation.cc:407] Connection header received
0807 18:11:57.169560 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.169565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.169680 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.169792 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.170264 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.171383 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.172926 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.173630 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.173765 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.174204 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.174350 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.174440 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.174579 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.174662 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 18:11:57.184185 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:57.182152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.8.62.188:44319)
0807 18:11:57.182376 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.182383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.182800 (+ 417us) server_negotiation.cc:407] Connection header received
0807 18:11:57.182938 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.182945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.183056 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.183150 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:57.183955 (+ 805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43160: BlockingRecv error: recv got EOF from 127.0.0.1:43160 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:11:57.257840 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.252318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.8.62.189:42791)
0807 18:11:57.252649 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.252655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.252893 (+ 238us) server_negotiation.cc:407] Connection header received
0807 18:11:57.253183 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.253188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.253264 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.253372 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.253867 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.254621 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.256040 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.256498 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.256688 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.257104 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.257218 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.257321 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.257471 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.257561 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260807 18:11:57.265992 32247 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 18:11:57.263960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.8.62.188:44319)
0807 18:11:57.264216 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.264223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.264661 (+ 438us) server_negotiation.cc:407] Connection header received
0807 18:11:57.264833 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.264838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.264915 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.265002 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 18:11:57.265750 (+ 748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43176: BlockingRecv error: recv got EOF from 127.0.0.1:43176 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 18:11:57.336055 32287 catalog_manager.cc:1259] Loaded cluster ID: 6dd85dd92ac143d2a14f47bef77d5c55
I20260807 18:11:57.336182 32287 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: loading cluster ID for follower catalog manager: success
I20260807 18:11:57.338755 32287 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: acquiring CA information for follower catalog manager: success
I20260807 18:11:57.340868 32287 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 16c0f799be4842b6b3aa7fef022f6347: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 18:11:57.402972 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.397692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.8.62.189:42791)
0807 18:11:57.397955 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.397962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.398177 (+ 215us) server_negotiation.cc:407] Connection header received
0807 18:11:57.398317 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.398321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.398397 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.398502 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.398999 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.399732 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.401182 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.401598 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.401696 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.402161 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.402282 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.402363 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.402610 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.402679 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 18:11:57.414815 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.408997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.8.62.188:44319)
0807 18:11:57.409228 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.409236 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.409508 (+ 272us) server_negotiation.cc:407] Connection header received
0807 18:11:57.409740 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.409747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.409955 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.410102 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.410510 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.411281 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.412620 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.413036 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.413174 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.413672 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.414193 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.414359 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.414426 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.414510 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 18:11:57.438542 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.422117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34872 (local address 127.8.62.189:42791)
0807 18:11:57.422396 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.422405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.429830 (+ 7425us) server_negotiation.cc:407] Connection header received
0807 18:11:57.430524 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.430535 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.430668 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.430755 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.431851 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.433430 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.435925 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.436651 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.436782 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.437657 (+ 875us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.437810 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.437917 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.438132 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.438230 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 18:11:57.441540 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.421593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.8.62.188:44319)
0807 18:11:57.421796 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.421802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.434178 (+ 12376us) server_negotiation.cc:407] Connection header received
0807 18:11:57.434872 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.434877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.434959 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.435072 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.435831 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.436642 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.439267 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.440012 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.440161 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.440734 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.440901 (+ 167us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.441031 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.441152 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.441236 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 18:11:57.442402 32419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 18:11:57.421518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.8.62.190:45165)
0807 18:11:57.434687 (+ 13169us) server_negotiation.cc:206] Beginning negotiation
0807 18:11:57.434694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 18:11:57.434716 (+ 22us) server_negotiation.cc:407] Connection header received
0807 18:11:57.434839 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 18:11:57.434845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 18:11:57.435080 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 18:11:57.435290 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 18:11:57.436033 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.437065 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.439540 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 18:11:57.440189 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 18:11:57.440295 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 18:11:57.440879 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 18:11:57.441411 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 18:11:57.441587 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 18:11:57.442075 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 18:11:57.442179 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13017,"thread_start_us":12885,"threads_started":1}
I20260807 18:11:57.447798 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 32021
I20260807 18:11:57.448990 32084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:57.560737 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 32021
W20260807 18:11:57.583401 32191 connection.cc:535] server connection from 127.0.0.1:43066 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 18:11:57.584082 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 32096
I20260807 18:11:57.585343 32160 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:57.693439 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 32096
I20260807 18:11:57.715411 8442 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu pid 32173
I20260807 18:11:57.716760 32236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 18:11:57.827759 8442 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskn0cCOk/build/asan/bin/kudu with pid 32173
2026-08-07T18:11:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4703 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9392 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (384781 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST